Methods, systems and devices relating to data storage interfaces for managing data address spaces in data storage devices

ABSTRACT

Data address management systems, methods, devices and uses for minimizing interaction with data consumers&#39; data on data storage devices, an embodiment comprising an external bus for communicatively interfacing the data storage system and data consumers; at least one storage medium components, each storage medium component comprising a plurality of storage locations having a unique storage location indicators; a translation layer module comprising a data address space having data addresses associable with storage location indicators; and a controller configured to store data in the storage locations and creating associations in the translation layer module between data addresses and the physical location indicators; wherein the data address space is accessible by the data consumer for addressing requests relating to data stored on the storage device and wherein the controller is configured to manipulate the arrangement of the data addresses in the data address space.

FIELD OF THE DISCLOSURE

The present disclosure relates to data storage systems and devices, andin particular, to methods, systems and devices relating to data storageinterfaces for managing data address spaces in data storage devices.

BACKGROUND

Storage devices, in general, have various methodologies for placing andallocating data onto the designated locations of the applicable storagemedium, depending generally on the type of storage medium. As such, moststorage media have an “address space” that may be used to associate aphysical “storage space” on that media and which address space ispresented to consumers of the data storage as the available storage.Depending on the storage media, there may be unique challenges andutilizations for managing the address space for the different types ofmedia.

For example, a typical flash memory device may in fact have 1.2 TB ofphysical storage, but only 800 MB of address space available for use bya data consumer; while described below in greater detail, this is tomanage a unique property of flash memory associated with the requirementthat prior to updating data in a memory cell that already has datatherein, the data must first be deleted (not overwritten) and anydeletion happens with reduced granularity than is possible by writes. Inother words, a complete erase block (which will be denoted as a “row” ofmemory cells in some examples) must be first deleted to write new datato any currently occupied/in-use memory cell in that erase block. Whensuch data to be deleted or updated co-exists in an erase block with livedata (which therefore cannot be erased), flash devices are presentedwith a problem which they have evolved in various ways to address.

In flash devices, a data address space is used to manage thisflash-specific issue, particularly in cases when (1) other memory cellswithin an erase block have data that must be maintained and (2) anincreased number of the available rows of memory have at least one cellwith live data. In such cases, data must be reallocated to ensure thatthere remain available rows of memory cells which can be entirelydeleted if necessary. The address space is configured to provide a dataaddress for data stored in memory blocks that are originally associatedwith memory blocks in a row (i.e. erase block), such row also containingother live and possibly non-related data, with memory blocks in otherrows that do not suffer the same limitation (meaning that data can bewritten thereto without deleting the entire row or that the remainingmemory blocks in the row can be deleted without destroying other livedata). In order to avoid the scenario in which there are no more rowsthat can be safely deleted, because all or nearly all rows contain atleast one block with live data, the flash device may be configured toconsistently re-associate the data in the data address space to preserveavailable rows; in some cases, the flash device is overprovisioned withmemory relative to the address space; in some cases, the flash device isalso overprovisioned with processing capability, for example with adedicated high-speed processor to manage the allocation of the data; insome cases, all or a combination of some of these are implemented.

While the above issue relates to flash memory devices, similartranslation layers may exist in other forms of data storage devices inwhich the physical locations, or addresses or other indicators of theactual location of storage locations within the storage media, may beabstracted by virtual addresses (which may be referred to herein as dataaddresses). For example, in spinning disk devices, the physical datalocations are frequently defragmented or collocated within the sametrack on a disk, or moved to an outer track (upon which data retrievalmay be faster), since data located in close proximity speeds performancein spinning disk media. In order to keep track of the data, atranslation layer, such as a register that maintains a log or tablekeeping track of data addresses and physical location indicators ismaintained. The data addresses exist in a data address space and, fromthe perspective of the physical storage media, are not currently managedin order to impact performance of the device.

From the perspective of the data consumer at any layer or abstraction(like for example, a user, a file system, or the forwarding tables on anetwork switch), all of this translation, and the underlying physicallocations are not visible; the data consumer views the storage device ashaving a capacity that is equal in size to that of the data addressspace, and in fact may in cases be indistinguishable from such dataaddress space. From the perspective of a data consumer, a flash device,for example, is a storage resource with a capacity that is the same asthe address space.

Flash memory is becoming a widely used storage media due to its improvedperformance in some respects: fast access speeds, low-power,non-volatile, and rugged operation. Most flash devices comprise a flashtranslation layer (FTL) that generally comprises an FTL driver thatworks in conjunction with an existing operating system (or, in someembedded applications, as the operating system) to make linear flashmemory, with erase blocks that are larger than individual write blocks,appear to the system like a single memory resource. It does that bydoing a number of things. First, it creates “virtual” small blocks ofdata, or sectors, out of the flash's large erase blocks. Next, itmanages data on the flash so that it appears to be “write in place” whenin fact it is being stored in different spots in the flash. Finally, FTLmanages the flash so there are clean/erased places to store data.

File systems, or other data consumers, such as, for example, operatingsystems on data consuming devices, such as DOS, typically use driversthat perform input and output in structured pieces called blocks. Blockdevices may include all disk drives and other mass-storage devices onthe computer. FTL emulates a block device. The flash media appears as acontiguous array of storage blocks numbered from zero to one less thanthe total number of blocks. In the example of DOS interacting with aflash memory device, FTL acts as a translation layer between the nativeDOS BPB/FAT file system and flash. FTL remaps the data to the physicallocation at which the data is to be written. This allows the DOS filesystem to treat flash like any other block storage device and remainsignorant of flash device characteristics. FTL appears to simply take thedata from the file system and write it at the specified location(sector). In reality, FTL places the data at a free or erased locationon the flash media and notes the real location of the data. It may insome cases also invalidate the block that previously contained theblock's data (if any), either in the FTL or in a separate data trackingmodule elsewhere on, or in data communication with, the FTL or thecomputing processor resources that is running the FTL. If the filesystem asks for previously written data, it requests the data at thespecified data address and the FTL finds and reads back the proper datafrom the actual physical storage location. Flash media allows only twostates: erased and non-erased. In the erase state, a byte may be eitherall ones (0xFF) or all zeroes (0x00) depending on the flash device. Agiven bit of data may only be written when the media is in an erasestate. After it is written to, the bit is considered dirty and unusable.In order to return the bit to its erase state, a significantly largerblock of flash called an Erase Zone (also known as an erase block) mustbe erased. Flash technology does not allow the toggling of individualbits or bytes from a non-erased state back to an erased state. FTLshields the file system from these details and remaps the data passed toit by writing to unused data areas in the flash media. This presents theillusion to DOS, or other data consumer, that a data block is simplyoverwritten when it is modified. In fact, the amended data has beenwritten somewhere else on the media. FTL may, in some cases, also takecare of reclaiming the discarded data blocks for reuse. Although thereare many types and manufacturers of flash memory, the most common typeis known as NOR flash. NOR flash, such as that available from IntelCorporation™, operates in the following fashion: Erased state is 1,programmed state is 0, a 0 cannot be changed back to a 1 except by anerase, and an erase must occur on a full erase block. For additionaldetail, the following document may be referenced: “Understanding theFlash Translation Layer (FTL) Specification”, Intel, 1998, and isincorporated herein fully by reference.

The emergence of commodity PCIe flash marks a remarkable shift instorage hardware, introducing a three-order-of-magnitude performanceimprovement over traditional mechanical disks in a single release cycle.PCIe flash provides a thousand times more random IOPS than mechanicaldisks (and 100 times more than SAS/SATA SSDs) at a fraction of theper-IOP cost and power consumption. However, its high per-capacity costmakes it unsuitable as a drop-in replacement for mechanical disks in allcases. As such, systems that have either or both the high performance offlash with the cheap capacity of magnetic disks in order to optimizethese balancing concerns may become desired. In such systems, thequestion of how to arrange data across both such media, but also withinsuch media is helpful in optimizing the requirements for wide sets ofdata. For example, in the time that a single request can be served fromdisk, thousands of requests can be served from flash. Worse still, IOdependencies on requests served from disk could potentially stall deeprequest pipelines, significantly impacting overall performance.

This background information is provided to reveal information believedby the applicant to be of possible relevance. No admission isnecessarily intended, nor should be construed, that any of the precedinginformation constitutes prior art.

SUMMARY

The following presents a simplified summary of the general inventive HIconcept(s) described herein to provide a basic understanding of someaspects of the invention. This summary is not an extensive overview ofthe invention. It is not intended to restrict key or critical elementsof the invention or to delineate the scope of the invention beyond thatwhich is explicitly or implicitly described by the following descriptionand claims.

A need exists for methods, systems and devices relating to memorystorage interfaces for managing data address spaces in data storagedevices that overcome some of the drawbacks of known techniques, or atleast, provide a useful alternative thereto. Some aspects of thisdisclosure provide examples of such methods, systems and devices. Someaspects further provide systems, methods and devices for amending dataaddresses within a data address space to arrange data addressesaccording to shared characteristics between underlying data andunderlying data of adjacent or near-adjacent data addresses.

In accordance with one aspect, there is provided a data addressmanagement system on a data storage device for storing data from a dataconsumer, the data storage system comprising: an external bus forcommunicatively interfacing the data storage system and one or more dataconsumers; at least one storage medium component, each storage mediumcomponent comprising a plurality of storage locations, each storagelocation having a unique storage location indicator; one or moretranslation layer modules each comprising a data address space having aplurality of data addresses, each data address being associable with atleast one storage location indicator; and one or more controllers eachconfigured to carry out computer readable instructions for writing andreading data in the storage locations and amending associations in theone or more translation layer modules between data addresses and thestorage location indicators; wherein the data address space isaccessible by the data consumer via the external bus for addressing datarequests to the one or more storage medium components and wherein atleast one of the controllers is configured to manipulate the arrangementof data addresses in the data address space by changing the associationsbetween data addresses and storage location indicators.

In accordance with another aspect, there is provided a method formanipulating a data address space of a data storage device, the datastorage device comprising at least one controller and at least onestorage medium component, the data address space comprising a pluralityof data addresses, each data address being associable with at least onestorage location indicator that is indicative of a storage location inthe at least one storage medium component, the method comprising thefollowing steps: selecting a first data address in the data addressspace associated with a first data via a first storage locationindicator, wherein the first data lacks a pre-determined storage-relatedcharacteristic with data associated with at least one data address thatis locationally-related to the first data address; selecting a seconddata address from the data address space, the second data address beingavailable for addressing data and being locationally-related to at leastone data address that is associated with a second data that shares thepre-determined storage characteristic with the first data; andgenerating an association between the second data address and thestorage location indicator indicating the current storage location ofthe first data.

In accordance with another aspect, there is provided a use of a dataaddress management system for minimizing interaction with the datastorage device by the data consumer by manipulating the data addressspace to arrange data addresses in the data address space in accordancewith at least one operational objective; the data storage system on adata storage device for storing data from a data consumer, the datastorage system comprising: an external bus for communicativelyinterfacing the data storage system and one or more data consumers; atleast one storage medium component, each storage medium componentcomprising a plurality of storage locations, each storage locationhaving a unique storage location indicator; one or more translationlayer modules each comprising a data address space having a plurality ofdata addresses, each data address being associable with at least onestorage location indicator; and one or more controllers each configuredto carry out computer readable instructions for writing and reading datain the storage locations and amending associations in the one or moretranslation layer modules between data addresses and the storagelocation indicators; wherein the data address space is accessible by thedata consumer via the external bus for addressing data requests to theone or more storage medium components and wherein at least one of thecontrollers is configured to manipulate the arrangement of dataaddresses in the data address space by changing the associations betweendata addresses and storage location indicators.

In accordance with another aspect, there is provided a data storagedevice interface between a data consumer and a data storage device, thedata storage device comprising (i) at least one storage medium componentwith storage locations and (ii) a data address space comprising aplurality of data addresses, the data storage device interfacecomprising: a functionality command for providing a data addressreallocation mode for manipulating data addresses in the data addressspace; a source data address identifier that identifies a first dataaddress for reallocation, the first data address in use for addressing afirst data and being associated with a first storage location indicatorindicative of the storage location storing the first data; and adestination data address identifier that identifies a second dataaddress, the second data address being available for addressing thedata; wherein the functionality command instructs the data storagedevice to allocate the first storage location indicator to thedestination data address.

In some aspects, the operational objectives may include any combinationof the following: defragmenting data addresses within one or more dataaddress space to minimize write requests required to write additionaldata objects to the data storage device; sequentializing data addresseswithin one or more data address spaces relating to data from asequence-dependent data object to minimize read requests required for asingle data object; grouping data addresses associated with temporallyrelated data to minimize read requests for multiple temporally-relateddata objects; grouping data addresses associated with similar metadatafrom multiple related data objects to minimize requests relating to anassessment of at least one shared characteristic of the multiple relateddata objects; grouping locationally-related data addresses in the dataaddress space when the data associated therewith via the storagelocation indicators are related metadata; as well as combinationsthereof.

In some aspects, the rearrangement of data addresses in the data addressspace may be accomplished by moving or changing physical locationindicators for data from a first data address to a second data addressthat is available for addressing data, wherein the second data addresscan is adjacent to or is otherwise locationally-related with in the dataaddress space at least one other data address that can be associatedwith data that shares the first similarity or storage-relatedcharacteristic with the data in the first data address.

In accordance with another aspect, there is provided a data storagesystem for managing data storage from a data consumer, the data storagesystem comprising: a storage medium defining a plurality of storagelocations, each one of which having a unique storage location indicatorassociated therewith; a translation layer defining a data address spacehaving a plurality of data addresses, each one of which being assignableto at least one said unique storage location indicator; a residentcontroller carrying out computer readable instructions for writing andreading data in said storage locations and managing data addressesassigned thereto in said translation layer, said resident controllerbeing further operable to carry out address space optimizationinstructions in reassigning at least some of said data addresses in saiddata address space to optimize access to said storage medium via saiddata address space; and an external communication interface accessibleto the data consumer to service data requests received therefrom to saidstorage medium via said data address space.

In general, the subject matter of this disclosure relates to themanagement of the data address space of data storage devices using theinfrastructure of such devices that have been developed to manage thephysical storage locations of data on the specific storage media thereonin accordance with specific operational limitations resulting from thecharacteristics of such media. The subject matter disclosed herein formanaging the data address space may have beneficial impact to either orboth of the data consumer and the data storage device (e.g. certainkinds of memory degrade at given storage locations as a function of thenumber of storage operations that occur at those locations and sominimizing read, write, or erase requests or operations will extend thelife of the storage device for the same utilization).

From the perspective of both the data consumer and the flash device, anefficiently managed address space can permit more efficient datatransactions (with, for example, lower latency) and a reduced number oftransactions to the physical storage cells of the flash (which willextend the life of the flash). Described are devices, systems, andmethods for organizing the placement of data within an address space,particularly for storage devices, so as to minimize the number ofoperations required to interact with that data.

The data consumer, in general, interacts with the data address space andthere may be advantages for managing the use of the data address space,as well as the physical data storage locations, via the internal. Theinstant disclosure leverages the inherent capabilities of many storagedevices, which utilize a translation layer for other reasons, to manageand/or organize the data address space in a manner that improvesinteraction between the data consumer and the storage device. Themanagement may also improve operational characteristics of the storagedevice, both in real-time or short-term operation, but also in long-termoperation. By manipulating the address space, the number of operationsrequired to interact with the data stored (or intended to be stored) onthe storage device can be reduced/minimized, made faster, and/orotherwise improved. Among other benefits, fewer data operations duringdata requests reduces bandwidth and latency, improves communicationsbetween the data consumer and the data storage device (as well as thecomponents and sub-components of the data storage device), and improvesthe lifetime of certain types of media (for example, the life time offlash may reduce over time or with increased usage).

Under load, what is frequently one of the larger performance pain pointsin enterprise data storage systems, is defragmenting the flash addressspace. Fragmentation is a general problem that will affect manyapplications of storage media devices, such as but not limited to PCIeflash. Defragmenting/compacting the Flash LBA using an FTL indirectiontable along with the appropriate functionality and interface to do thisredirection from the host could provide performance enhancements in somecases for, as an example, sustained, mixed-write workloads.

Other aspects, features and/or advantages will become more apparent uponreading of the following non-restrictive description of specificembodiments thereof, given by way of example only with reference to theaccompanying drawings.

BRIEF DESCRIPTION OF THE FIGURES

Several embodiments of the present disclosure will be provided, by wayof examples only, with reference to the appended drawings, wherein:

FIG. 1 is a diagrammatic representation of a data consumer, atranslation layer, and the physical storage locations of a physicalstorage medium;

FIG. 2 is a representative diagram of a data address management systemin use with a data consumer device in accordance with one embodiment ofthe instantly disclosed subject matter;

FIG. 3 is an exemplary and illustrative representation of a translationlayer and physical storage medium in accordance with one embodiment ofthe instantly disclosed subject matter;

FIG. 4 is an exemplary representation of a single data address space atdifferent stages of use;

FIG. 5 is a representative diagram of another data address managementsystem in use with a data consumer device in accordance with oneembodiment of the instantly disclosed subject matter; and

FIG. 6 is a block diagram representative of a method for managing a dataaddress space in accordance with one embodiment of the subject matterdisclosed herein.

DETAILED DESCRIPTION

The present invention will now be described more fully with reference tothe accompanying schematic and graphical representations in whichrepresentative embodiments of the present invention are shown. Theinvention may however be embodied and applied and used in differentforms and should not be construed as being limited to the exemplaryembodiments set forth herein. Rather, these embodiments are provided sothat this application will be understood in illustration and briefexplanation in order to convey the true scope of the invention to thoseskilled in the art.

In many modern storage devices, such as flash storage devices and diskdrive storage devices, as well as, for example, the combination of RAM,MMU and a CPU in a general-purpose computer, there are provided variousways of managing the placement of data within the physical storage mediaof such devices. Such schemes may be implemented in order to place, moveand/or manage the data within locations on the physical storage media toovercome data storage limitations associated with such devices, whereinvisibility to the data consumer of such placement, movement, and/ormanagement to overcome said limitations may be minimized. To implementsuch data management schemes, many such devices include dedicated and/orhigh-speed device-internal components and functionalities, such asprocessors, data buses, and memory for facilitating rapid placement,movement and management of the data. A translation layer is also oftenused to integrate data addresses, visible to a data consumer, andindicators of the location of the corresponding data on the physicalstorage medium, which are not visible to the data consumer. Suchtranslation layers permit the device to move data storage locations onthe physical data storage media without necessarily changing the dataaddress of that data, which is used by a data consumer to transact withthe data or applicable data storage (e.g. send read, write or updatedata requests). Subject matter disclosed herein exposes thosefunctionalities and/or components to non-device data consumers(including, for example, users, other computers or computing devices,applications, and other APIs). This may facilitate, inter alia: (1) theability to offload the overhead of data transfer between physicallocations from data consumers directly onto the device-internal bus andtransfer compute resources; and/or (2) the explicit manipulation of thetranslation layer (including the data address space) by off-device dataconsumers and/or (3) the use of data-specific characteristics to moreoptimally manage either or both of the data address space and storagelocation without the associated overhead of sending and receiving datarequests between data consumer and device. By exposing certain controlsof the device-internal components and the applicable translation layer,a data consumer can manipulate both the data address space and/or thelocations of the physical data storage medium used for a given data setto, for example, optimize the use, operation, and useful life expectancyof the data storage device. These functionalities have heretofore beenunavailable, or to the extent that they have been possible, requiresignificant additional compute and network resources (i.e. a read datarequest, transmission of the requested data over the network in responseto the read request, some degree of data storage and data processing atthe requesting data consumer, a write request of the applicable databack over the network, and, in some cases, an acknowledgement of thewrite back to the data consumer). Myriad other advantages, many of whichwill become apparent from the subject matter disclosed herein, shallbecome apparent.

As an illustrative example of a device-specific storage limitation, thefollowing example provides an overview of a limitation associated with aflash memory device (which may include any non-volatile memory device).Many modern flash devices utilize specific device-internal hardware anda flash translation layer to overcome an inherent storage limitationassociated with flash data storage. In many modern flash memory systems,for example, flash memory media can only be written to memory units(e.g. a data block) if such units are in an erased state; such memoryunits in flash memory media, however, are generally only erasable inblocks of a plurality of such memory units. As such, a device-specificdata storage management regime must be implemented in order to accountfor updating data stored in a subset of memory units in an erase blockwhen other memory units in the same erase block cannot or should not beerased.

In general in most flash devices, if a data block needs to be updated orchanged, it must first be erased; the granularity of a write block,however, is smaller than an erase block. As such, in order to update,change or over-write a given data block, all other data blocks that area part of the same erase block must also first be erased, wherein anerase block consists of all data blocks in the flash memory media thatare erased at the same time or as a result of the same erase command orset of erase commands. Other data blocks in the erase block may storeunrelated data, which may not have to be updated (in fact, it is oftenboth possible and likely that such data must not be changed in order tomaintain reliable data storage). In order to account for thislimitation, flash devices use a flash translation layer to associatedata addresses, which may be used by data consumers to identify andlocate data, with a location indicator representing the data block onthe physical storage medium. When existing data is updated with newdata, for example, but other data blocks in the erase block may not beerased, the flash device will write the new data to a further data blockthat is already in an erased state and the flash translation layer willbe updated to now associate the data address with said further datablock rather than the original data block that stores the pre-updatedversion of the data. The original data block, although not in an erasedstate and containing the older version of the data, will hold stale (or“dead”) data and may be erased without losing current or live data(which may or not be true for at least one other data block in the eraseblock). Since data in data blocks within the same erase block may beupdated, changed or over-written in an arbitrary manner over time, atleast with respect to one another, erase blocks can become sparselypopulated with current data over time. The number of data addressesavailable in such devices are typically less than the actual number ofavailable data blocks in order to account for this limitation (in manydevices, the data storage medium is overprovisioned relative to thenumber of available data addresses). Moreover, the sparsely populatederase blocks must periodically be consolidated; otherwise, the devicewould reach capacity in an arbitrary manner, wherein some portion of thestorage medium would in fact be available for storage as it holds deaddata, which could be erased if the entire erase block could also beerased.

To mitigate this problem, the device periodically consolidates datablocks from a plurality of sparsely populated erase blocks into asmaller number of available erase blocks. Using the information in theflash translation layer, and the device-internal processor(s), databus(es), and cache memory(ies), the sparse erase blocks are configuredto periodically perform “garbage collection” by moving or copying datafrom data blocks from a plurality of sparsely populated erase blocks andconsolidating such data blocks onto a smaller number of more denselyused erase blocks so that one or more of the sparsely populated eraseblocks from which the data was previously stored can be erased, therebyfreeing up space for data storage. Flash devices must perform this typeof garbage collection as the device approaches the maximum number ofavailable erase blocks that have at least one non-erased data block,otherwise the flash device will not be able to accept further data forstorage. Such garbage collection places significant strain on theprocessing capabilities of the device and data requests during suchtimes may experience increased latency and/or reduced throughput, forexample. Moreover, it results in assigning data addresses to data in anarbitrary manner. In general, with higher over-provisioning, thisgarbage collection needs to take place with reduced frequency, therebyreducing the processing strain but increasing the overall cost of thedevice. Similarly, with higher speed processing power, data buses, andmemory (e.g. SDRAM), the garbage collection can occur more quickly andmore frequently with less reduction on performance, which also meansless over-provisioning is required but also with increased cost. Theperformance and cost of these solutions are generally balanced invarious manners and may depend on, among other factors, the device, themanufacturer thereof, the data consumers that use or may use the device,and the intended use or objective of the device.

In another device type, spinning disk data storage devices, anotherdevice-specific storage limitation exists which utilizes similardevice-internal processing components (processor, bus, memory) and atranslation layer for managing data placement on the physical storagemedium with little or no visibility to the data consumer. Portions ofspinning disks are subject to failure; such failure may include theoccurrence of bad sectors or regions on a disk, typically as the diskages. To account for such occurrence, a translation layer and thedevice-internal components associate a different block, sector orportion of the disk that has not failed with the same data address inthe translation layer as the failed block, sector or portion, and, ifnecessary, place a copy of the data into the new location. The dataconsumer accesses the data (or the associated data storage, if forexample it is writing new data) according to same data address and hasno visibility to the fact that there is a different location for storageon the physical storage medium. Over time, the data address space maybecome sparse and exhibit an arbitrary relationship to the correspondinglocations on the physical storage medium or the data itself. As anexemplary result of this device-specific limitation, when related data,which may be read at same or similar times, is located in non-contiguouslocations of a spinning disk, significant access times are introducedsince the armature mechanism needs to move to appropriate location.Furthermore, as the data address becomes more sparse over time, fewercontiguous data addresses become available, and instead of associatingan entire file, for example, with single contiguous set of dataaddresses (which would require a single or fewer number of datarequests), a sparse data address space requires associating data fromthe same set of related data with a number of distinct data addresses,each of which requiring its own data request or set of data requests toaccess the data storage resource. This problem, which is applicable toflash devices and other devices, is exacerbated on a spinning disk when,for example, the armature mechanism must be displaced to access thearbitrarily located physical locations associated with the sparse dataaddresses. By exposing, as provided for herein, control over both thedata address space management, as well as the location management of thephysical storage media, the speed of the device in responding to datarequests, there is increased efficiency of data requests, and theoptimal placement of data within physical storage according to datacharacteristics.

In another example of a device-specific limitation, computing systemsgenerally comprise a CPU, which processes an instruction stream. The CPUalso generally comprises (although it may form a separate componentaccessible via a data bus or other communicative coupling) a memorymanagement unit (“MMU”), which may provide similar or analogousfunctionalities as the translation layer. The MMU maintains a mappingbetween virtual addresses of data, which is visible to theCPU/instruction stream, and a location address corresponding to thephysical location on the physical RAM. Both the location addressmanagement and the virtual address space in the MMU, and associationstherebetween, are controlled and/or managed by the computing systemoperating system. As such, in some embodiments, RAM storage locationmanagement, as well as management of the virtual address space can beimplemented according to the methods presented herein. That is, dataaddress space management and physical location management, normally thedomain of the OS and MMU, are exposed to the data consumer.

As a result of these problems, memory storage devices have developedboth translation layer abstractions to internally manage the arrangementof physical storage locations on the storage media within a data storagedevice. In addition, internal overprovisioning of storage media relativeto the size of the data address space, as well as higher-performinginternal components (e.g. an internal bus on the data storage devicethat is faster, similar to a network switch forwarding plane, possiblyowing to its dedicated functionality) that can be dedicated to themigration and rearrangement of data stored on the physical storage mediahave evolved to overcome device-specific storage limitations.Embodiments hereof utilize such device-internal components andtranslation layer to more optimally interact with and utilize saiddevice.

As used herein, a “computing device” may include virtual or physicalcomputing devices, and also refers to devices capable of receivingand/or storing and/or processing and/or providing computer readableinstructions or information. Embodiments of the instantly disclosedsubject matter may be implemented in a variety of different arrangementsor combinations of computing devices and/or components thereof withoutdeparting from the scope and nature of the subject matter describedherein; as described herein, reference to a single computing device mayinclude a combination of multiple computing devices and/or componentsthereof in a connected and/or distributed manner.

As used herein, “memory” may refer to a resource or medium that iscapable of having information stored thereon and/or retrieved therefrom.Memory, as used herein, can refer to different components, resources,media, or combinations thereof, that retain data, including what may behistorically referred to as primary (or internal or main memory due toits direct link to a computer processor component), secondary (externalor auxiliary as it is not always directly accessible by the computerprocessor component) and tertiary storage, either alone or incombination, although not limited to these characterizations. Althoughthe terms “data storage”, “storage” and “memory” may sometimes carrydifferent meaning, they may in some cases be used interchangeablyherein.

As used herein, memory may comprise a single medium or unit, or it maybe different types of resources that are combined logically orphysically. This may include memory resources that provide rapid and/ortemporary data storage, such as RAM (Random Access Memory), SRAM (StaticRandom Access Memory), DRAM (Dynamic Random Access Memory), SDRAM(Synchronous Dynamic Random Access Memory), CAM (Content-AddressableMemory), or other rapid-access memory, or more longer-term data storagethat may or may not provide for rapid access, use and/or storage, suchas a disk drive, flash drive, optical drive, SSD, other flash-basedmemory, PCM (Phase change memory), or equivalent. A data storage mediumor resource may include, in whole or in part, volatile memory devices,non-volatile memory devices, or both volatile and non-volatile memorydevices acting in concert. Other forms of memory storage, irrespectiveof whether such memory technology was available at the time of filing,may be used without departing from the spirit or scope of the instantdisclosure. For example, different high-throughput and low-latencystorage media can be used in the same manner as PCIe Flash, includingdifferent solid-state memory technologies that will appear on the PCIebus. Technologies including phase-change memory (PCM), spin-torquetransfer (STT) and others that may develop can be used as memory storagemedia components. Some storage resources can be characterized as beinghigh- or low-latency and/or high- or low-throughput and/or high- orlow-capacity; in many embodiments, these characterizations are based ona relative comparison to other available storage resources on the samedata server or within the same distributed storage system. For example,in a data server that comprises one or more PCIe Flash data storagecomponents as well as one or more spinning disks, the PCIe flash will,relative to other storage resources, be considered as being lowerlatency and higher throughput, and the spinning disks will be consideredas being higher latency and higher throughput. Higher or lower capacitydepends on the specific capacity of each of the available storageresources, although in embodiments described herein, the form factor ofa PCIe flash module is of lower capacity than a similarly sized formfactor of a spinning disk. It may include a memory component, or anelement or portion thereof, that is used or available to be used forinformation storage and retrieval.

A “computing processor component” refers in general to any component ofa physical computing device that performs arithmetical, logical orinput/output operations of the device or devices, and generally is thecomponent of the computing device that carries out instructions. Thecomputing processor component may process information for a computingdevice on which the computing processor component resides or for othercomputing devices that are communicatively coupled (both physical andvirtual). It may also refer to one or a plurality of components thatprovide processing functionality of a computing processor component, andin the case of a virtual computing device, the computing processorcomponent functionality may be distributed across multiple physicaldevices that are communicatively coupled. A computing processorcomponent may alternatively be referred to herein as a CPU or aprocessor.

As used herein, “priority” of data generally refers to the relative“hotness” or “coldness” of data, as these terms would be understood by aperson skilled in the art of the instant disclosure. The priority ofdata may refer herein to the degree to which data will be, or is likelyto be, requested, written, or updated at the current or in an upcomingor given time interval. Priority may also refer to the speed at whichdata will be required to be either returned after a read request, orwritten/updated after a write/update request. In some cases, a highfrequency of data transactions (i.e. read, write, or update) involvingthe data in a given time period, will indicate a higher priority.Alternatively, it may be used to describe any of the above states orcombinations thereof. In some uses herein, as would be understood by aperson skilled in the art, priority may be described as temperature orhotness. As is often used by a person skilled in the art, hot data isdata of high priority and cold data is data of low priority. The use ofthe term “hot” may be used to describe data that is frequently used,likely to be frequently used, likely to be used soon, or must bereturned, written, or updated, as applicable, with high speed; that is,the data has high priority. The term “cold” could be used to describedata that is infrequently used, unlikely to be frequently used, unlikelyto be used soon, or need not be returned, written, updated, asapplicable, with high speed; that is, the data has low priority.Priority may refer to the scheduled, likely, or predicted forwarddistance, as measured in time, between the current time and when thedata will be called, updated, returned, written or used. Different dataor data objects that have similar priority at similar times aretemporally related.

As used herein, the term “data consumer” may refer to a computing deviceor software running thereon, virtual or physical, which generates orconsumes data and/or data objects that are stored or intended to bestored on a data storage device. The data consumer is, in someembodiments, a logical or physical entity that is capable of, orconfigured for, making and/or sending data requests to the data storagedevice in respect of data stored thereon. Without limitation, but asillustrative examples, a data consumer may refer to a file system, adata client, an operating system, the computing device or computerprogram that runs such file systems, an application or anapplication-layer process, an operating system, a user thereof, or acombination of any of the foregoing. In some cases, web browsers may beconsidered to be data consumers that connect to, for example, webservers which retrieve web pages for display at the end user's computer;email clients retrieve email from mail servers, and as such may beconsidered to be a data consumer. Data consumers may communicate withdata storage devices across physical networks which comprise theInternet, or they may reside on a single computing device or a group ofcommunicatively connected computing devices. In accordance with the OSImodel of computer networking, data consumers and data storage devicesmay be connected via a physical network of electrical, mechanical, andprocedural interfaces that make up the transmission. They may utilizedata link protocols to pass frames, or other data link protocol units,between fixed hardware addresses (e.g. MAC address) and utilize variousprotocols, including but not limited to Ethernet, Frame Relay,Point-to-Point Protocol. Data consumers and data storage devices mayalso communicate in accordance with packetized abstractions, such as theInternet Protocol (IPv4 or IPv6) or other network layer protocols,including but not limited to Internetwork Packet Exchange (IPX), RoutingInformation Protocol (RIP), and Datagram Delivery Protocol (DDP).End-to-end transport layer communication protocols may be utilizedwithout departing from the scope of the instant disclosure (suchprotocols may include but not limited to the following: AppleTalkTransaction Protocol (“ATP”), Cyclic UDP (“CUDP”), Datagram CongestionControl Protocol (“DCCP”), Fibre Channel Protocol (“FCP”), IL Protocol(“IL”), Multipath TCP (“MTCP”), NetBIOS Frames protocol (“NBF”), NetBIOSover TCP/IP (“NBT”), Reliable Datagram Protocol (“RDP”), Reliable UserDatagram Protocol (“RUDP”), Stream Control Transmission Protocol(“SCTP”), Sequenced Packet Exchange (“SPX”), Structured Stream Transport(“SST”), Transmission Control Protocol (“TCP”), User Datagram Protocol(“UDP”), UDP Lite, Micro Transport Protocol (“μTP”). Such transportlayer communication protocols may be used to transport session,presentation- or application-level data. Other layer protocols and othertypes of data protocol units, as would be understood by a person skilledin the art and science of the subject matter disclosed herein, may beused for the communication of data requests and responses theretobetween data consumers and data storage devices.

As used herein, a “data request” may refer to instructions or requestsrelating to the reading, writing, updating, and/or calling of data; andsuch data transactions may comprise of: (i) read requests and writerequests that are issued by data consumers requesting an action be takenwith specific data (e.g. read, write, update), (ii) responses to suchread and write requests that are returned by data servers or datastorage devices in response to a data request, and (iii) the read/writerequests and their associated responses taken together. In someembodiments, data requests may originate at data consumers; in someembodiments, they may originate from applications running on or at adata consumer. In some embodiments, data requests are sent to dataservers and then responded to appropriately, and a response is returnedto the data client. In some embodiments, data requests may beasymmetrical in that a write request generally carries a relativelylarge amount of data from a data client to the distributed data storagesystem, since it must include the data to be written, and the datastorage system returns a relatively much smaller response thatacknowledges receipt and confirms that the data was written to memoryand possibly information relating to the data address associated withthe data; in some embodiments, a read request is a relatively smallamount of data, whereas the response to the read request from the datastorage system is the data that was read and is therefore much largerthan the request, relatively speaking. Data requests are often made inaccordance with an application or session layer abstraction; in someembodiments, they are instructions from one computing device (or otherendpoint) to implement an action or a subroutine at another computingdevice. In some embodiments, data requests are sent over the network asNFS requests (application layer) contained within TCP segments(endpoint-to-endpoint data stream) which in turn are carried in IPpackets over the Internet, across Ethernet-based devices within framesacross networking devices. Other exemplary data requests may form RPC(Remote Procedure Call) requests, which may in turn comprise NFSrequests or other types of data requests. Other examples include iSCSI,SMB, Fiber Channel, FAT, NTFS, RFS, as well as other file systemrequests and responses which would be known to persons skilled in theart of the instant disclosure. In some embodiments utilizing NFS, an NFSrequest and its corresponding response, would each be considered a datatransaction.

In some embodiments, data consumers and data consumer applicationscommunicate with data storage devices to access data resources inaccordance with any of a number of application-level storage protocols,including but not limited to Network File System (“NFS”), Internet SmallComputer System Interface (“iSCSI”), and Fiber Channel. Other storageprotocols known to persons skilled in the art pertaining hereto may beused without departing from the scope of the instant disclosure.Additionally, object storage interfaces such as Amazon's S3,analytics-specific file systems such as Hadoop's HDFS, and NoSQL storeslike Mongo, Cassandra, and Riak are also supported by embodimentsherein.

In one embodiment of the subject matter disclosed herein there isprovided a data address management system on a data storage device forstoring data from a data consumer, the data storage system comprising:an external bus for communicatively interfacing the data storage systemwith one or more data consumers; at least one storage medium component,each storage medium component comprising a plurality of storagelocations, each storage location having a unique storage locationindicator; one or more translation layer modules each comprising a dataaddress space having a plurality of data addresses, each data addressbeing associable with at least one storage location indicator; and oneor more controllers each configured to carry out computer readableinstructions for writing and reading data in the storage locations andamending associations in the one or more translation layer modulesbetween data addresses and the storage location indicators; wherein thedata address space is accessible by the data consumer via the externalbus for addressing data requests to the one or more storage mediumcomponents and wherein at least one of the controllers is configured tomanipulate the arrangement of data addresses in the data address spaceby changing the associations between data addresses and storage locationindicators.

In some embodiments, the data storage device is a device that is capableof interfacing with a data consumer and is, in whole or in part, usedfor storing data. It may refer to a flash memory device, such as Intel'sPCIe flash product, a hard drive, or other types of data storagedevices. In general, such devices will comprise of memory components, aprocessing component, and a translation resource. The external bus is,in general, a computing interface that permits data consumers tocommunicate with the data storage device.

The storage medium components generally comprise of the component of thedata storage device where the data is physically stored. It may include,but is not limited to, the spinning magnetic disk and armaturecomponents (for reading and writing to the disks), flash memorycomponent (e.g. NAND flash), or other types of storage media on whichcomputer readable data can be stored. In general, specific locations onthe storage medium components are capable or configured to store adiscrete unit or portion of data, such locations are addressable by thestorage device by having a storage location indicator associatedtherewith. For example, in FIG. 1, the storage medium 112 (in this caseNAND flash) is divided into R/W blocks 114, each having a uniqueidentifier that is used to identify that storage location when a requestfor data that is or is to be written/read/updated on that block isreceived by the data storage device from a data consumer 110.

The translation layer is a physical or logical resource that permits thestorage device to generate a level of abstraction above the physicallocation indicators. Referring to FIG. 1, which shows a diagrammaticrepresentation of a data consumer in communication with a storage device100, the translation layer is the FTL (or Flash Translation Layer) 120,which is a table, log, or other form of register, that associates a setof virtualized data addresses to physical location indicators. Each dataaddress is virtually assigned to a physical storage location byassigning to the data address in the translation layer the locationindicator unique to that physical storage location. As such, each dataaddress is a virtual or logical representation of an address to aphysical storage location and is representative of a unit of storagethat has the same capacity as the associated physical storage locations.As such, a data address space, which is the set of all available dataaddresses in the translation layer, is a virtualized representation ofsome or all of the storage capacity of the data storage device. In someembodiments, while the actual storage capacity in the storage mediumcomponents may be different from that HI logically presented by thetranslation layer, the storage capacity as represented by the dataaddress space is the capacity that is made available by the data storagedevice. In one exemplary embodiment, which utilizes the hardware from anIntel™ PCIe flash device, the data address space represents 800 GB ofstorage, even though the flash storage media in the device in fact holds1.2 Tb of data storage. In some cases, the data addresses that make upthe data address space are a set of consecutive data addresses (whichcan be represented according to different addressing conventions). Inother embodiments, not all the data addresses are consecutive; there maybe distinct sub-groups of addresses that are consecutive; alternatively,there may be groups of related addresses for which single or a reducednumber of data requests can efficiently interact with in order to accessthe data underlying the data addresses. In yet other embodiments, theremay be a further level of abstraction above the translation layer, suchas but not limited to data storage objects; such data storage objectsmay be presented by the data storage device as virtual or logical“containers” for data and will, in some embodiments, be considered asthe data address space.

The controller may be a computing processor that is capable of carryingout instructions to implement actions within the data storage device. Itmay be configured to execute instructions for forwarding data, copyingdata, moving data, updating data, erasing erase blocks, and programmingstorage blocks in the storage medium component. It may be configured tocarry out more complex series of instructions in response to commandsand/or acquired or measured data. In general, it may consist of a CPU orother computer processing unit. In addition, the controller may beconfigured to manipulate or manage the data address space by changingassociations within the translation layer, or it may cause the datastorage device to move data from one data storage location to anotherand then cause the new location to be associated via its storagelocation indicator within the data address space such that the new dataaddress meets one or more operational objectives, particularly withrespect to those data addresses that are locationally-related (by virtueof being adjacent, nearly adjacent, sequential, or in predetermined ordeterminable locations).

The operational objectives may, in some embodiments, be related toorganizing data addresses within the data address space in order toplace multiple data addresses, which relate to data that share one ormore similarities, in adjacent or nearly adjacent positions in the dataaddress space. These similarities may in some embodiments relate to thedata that is (or will be) stored at the physical location whose uniquelocation indicator is associated with one or more of the multiple dataaddresses; this is intended to include the lack of data (or associatedphysical location indicator) or lack of live data, associated with adata address that may provide the similarity therebetween. In someembodiments, data addresses that are either (a) associated with locationindicators that are erased, empty, or are eligible for being erased(because, for example, the data therein is no longer needed or current);or (b) not associated with any physical location indicators, are deemedto share a similarity and the controller may, in respect of certainoperational objectives, arrange these data addresses together as much aspossible within the data address space. When writing data from largedata objects, the number of write requests will be reduced if there is asingle or a small number of contiguous data addresses available for thatdata. In some cases, the similarity may be that the end of the dataassociated with a first data address is the beginning of the data ofanother piece of data that is associated with a second address (in thiscase, the two pieces of data may be sequential in a data stream or asthey exist in a data object. For example, a video or audio file that isread sequentially will require fewer reads, and less work by the networkelements and/or data consumer, if the data is available to be read insequential data addresses. A single or reduced number of read requestscan be used since an entire in-order sequence of data can be retrievedby memory; in such a case, the number of data requests is limited onlyby the size of the data requests, rather than the number of discretegroups of data addresses that are associated within the translationlayer within that object. For data objects or data streams that are notsequence-dependent, mere contiguousness of the associated data addressesare sufficient. In some cases, the similarity will be that theunderlying data associated with the multiple data addresses is that thedata constitutes metadata, or is capable of being used to calculate orassess metadata or other such data; instead of writing a copy of suchdata to a specific and separate file, which is what is generally done inthe art, the data addresses associated with the metadata can be grouped,thereby permitting for efficient data access to that data (with aminimized number of interactions) without generating additional copies(which consumes capacity and may become out-of-date quickly). Othersimilarities may include a temporal relationship; for data objects thattend to be associated with data requests made or communicated at orclose to the same time, grouping such data according to their dataaddresses in the data address space can provide a form of tiering that,for example, can permit for pre-fetching data from temporally relateddata objects such that reading or writing such data can be accomplishedin a reduced number of data requests.

In some embodiments, data addresses are associated with data from dataobjects because the data is written at a location that is associated viaa physical location indicator with that data address. In other cases,the data address has been associated with a physical location becausethe physical location has been designated or has been made available forsuch data to be written. As such, data addresses may be associated withdata that is to be part of a write request, even if the data has notbeen stored in the data storage device described herein.

In some embodiments, the disclosed subject matter relates to interfacesto the data storage device by the data consumer, which cause moving andcopying data to a new physical location, which creates a new physicallocation indicator that can be associated with any data address in thedata address space. It may also comprise copying or moving just thephysical location indicator to another address space. In some cases, thedata consumer must be notified that the data address associated with thedata has been changed so that future data requests that relate to thatdata are kept up to date by the data consumer.

In some embodiments, the data address space may comprise avirtualization of data storage that appears to a data consumer as singlestorage targets wherein the data addresses comprise consecutiveaddresses that range from, for example, bit 0 to Mb 800 in 4 Kbsegments, with an address pointing to each segment or block. In suchembodiments, the data address space may be managed by ensuring that,depending on the storage objective, the number of contiguous availabledata addresses within the data address space is maximized (even thoughthe data addresses may or may not point to storage locations which haveany locational relationship at all). Other arrangements in such dataaddress spaces include senquentializing consecutive data addresses inthe same order as the data to which they have been associated in thetranslation layer; grouping a number of data addresses, which relate totwo or more data objects, together contiguously when such data objectsare likely to be (or have in the past been) temporally related, in thatif requests are received relating to any one of the data objects in thegroup, data requests to the other data objects are likely to be receivedand so data associated with both data objects can be read concurrentlyor with reduced number of read requests; grouping in a contiguousmanner, or in predetermined locations with the data address space, dataaddresses that are related to data that constitutes or is capable ofdetermining metadata of a group of related data objects.

In some embodiments, the data address space is not necessarily acontiguous and/or consecutive group of addresses that appear torepresent virtualized consecutive storage locations. The data addressspace can be sub-groups of data addresses which are not necessarilyconsecutive with one another even if the data addresses within eachsub-group are consecutive. For example, in some embodiments a dataaddress space may comprise a plurality of separate blocks or groupingsof data addresses wherein each grouping may represent a specific storagetarget, such as an assigned physical or virtual data storage resource.In such a case, there may be a set of data addresses associated with afirst resource or group of resources, and additional data addressesassociated with additional resources (whether physical or virtualized);the data address space may comprise multiple sets of data addresses,wherein such sets of data addresses are non-contiguous and otherwisenon-related with respect to each other.

In other cases, the data address space may comprise one or more datastorage objects. In some storage devices, instead of presenting a dataaddress space as consecutive data addresses, the storage devices may beconfigured to present storage objects as a data address space; inconjunction with such a device, the data address space of an embodimentof the instant disclosure would be the one or more data storage objectspresented by the device that the device maps to the physical locations,or utilizes in conjunction with an additional layer of abstraction tomap the data storage objects to data sub-addresses, such datasub-addresses being associable with physical storage locationindicators. In such a case, the data consumer could over time, have anumber of storage objects that comprises non-contiguous portions of thestorage objects that are associated with related data (e.g. the datacomes from the same data object, they come from related data objects, orthey are related metadata, although other examples are possible in thesame manner as described as for other data address spaces herein); inaddition to being non-managed data address space (by being, for example,non-contiguous, or out of sequence, or duplicated metadata, or otherexamples of non-managed data address space) within a storage object, theportion of the data address space may be found across different storageobjects within the data address space. Rather than forcing the dataconsumer to request all of the storage objects and then re-writing thedata objects into storage objects more efficiently, some embodiments ofthe instant disclosure utilize a re-mapping command, which may inembodiments comprise a move or a copy command, provide for an interfacethat causes such storage object devices to re-arrange the data addressspace, include management of the storage objects. Such an embodiment maypermit the system to write data relating to metadata into specific dataaddresses as part of certain or all types of interaction with dataobjects for which metadata will be required. As such, the metadata willbe up-to-date and stored in association with easily identifiable dataaddresses; in other cases, the data required to calculate the necessarymetadata may also be stored in association with easily identifiable dataaddress. Some non-limited examples of assessed metadata may include, forillustrative purposes only, time of last access, identity of last useraccess, etc.

In some embodiments, the data address space may comprise a number ofdata storage objects, and may not include a direct mapping between dataaddresses accessible or visible to the data consumer, but rather themapping of data from within a data storage object is handled by the datastorage device; rather than sending data requests to a data address, adata consumer would send a get or put command, for example, to a datastorage object ID. An exemplary command may be as follows:get{data_object_ID, [the_data]}, where data_object_ID is the data objectID that allows the data storage device and the data consumer to agree onwhich data object in respect of which they are interacting and[the_data] is the data that is to be read (which in the case of a read,may be a null set), and alternatively, another command may beput{data_object_ID, [the_data]}, which would write [the_data] to a dataobject that can be identified as data_object_ID. The data address spaceincludes the set of all data objects made available by the data storagedevice and could be considered a “per-object” data address space ratherthan a “per-address” data address space. In yet further embodiments, thedata storage device may be an aggregation of other disks (or datastorage devices), each of which having their own controller, translationlayer, and storage medium components; in this example, the controllerson the subset of aggregated data storage devices (which makes up thedata storage device) can communicate with one another. Over time, forexample, one or more data storage objects can become sparse orfragmented, and the data address space management subject matterdisclosed herein may be configured to manage the data address spaceassociated with such data storage objects. If a number of data storageobjects become sparse over time, it would be helpful to both the dataconsumer and the data storage device if the sparse data sets could becombined into a single data storage object and the management of thatdata address space were handled by the data storage device (which inthis case, is a number of aggregated data storage devices, such asJBODs—“Just A Box of Disks”—or JBOKs—“Just A Bunch of Kinetics”). As apractical example, in some embodiments, a data consumer may wish tostore a set of cold data to the set of JBODS (which, taken together, arereferred to in this example as the data storage device and can bereferred to as “sub-devices”), wherein such sets of cold data eachcomprise a plurality of unrelated pieces of data. Over time, as a numberof these data objects containing cold data are accessed and changed, thedata within a data object becomes sparse, but an entire data object mustbe called when reading any data therefrom. As such, there is providedherein a mechanism for re-arranging data from across a number of thesedata objects into a single or reduced number of data objects, therebyreducing the number of data requests associated with interacting withthat data. In this example, there are multiple controllers, i.e. one oneach sub-device, which can communicate with one another across thesub-devices to re-allocate, read from, or write to, the storage mediumcomponents of the other sub-devices. Other use cases may includeexamples wherein data that is temporally-related, with respect to whensuch data is read, written, or updated, is placed within the same datastorage object; data that is associated with related metadata for agroup of similar or related data objects is placed within the datastorage object.

With reference to FIG. 5, there is provided an example of a data storagedevice 500 which presents a data address space as a set of data storageobjects and comprises of a number of sub-devices 520, 530, 580. There isa common data address space, comprising of a data storage objecttranslation layer for data storage objects 510, even though eachsub-device may have its own translation layer 522, 532, 582 referring toits own storage medium component 521, 531, 581. Each sub-device alsocomprises its own controller 523, 533, 583, each of which maycommunicate with one another when amending associations between a firstdata in a first data object, to the same first data into a second dataobject, for example. In some embodiments, each sub-device may have aunique IP address, even though the data storage object address spacespans the entire aggregation of the sub-devices. In the example shown inFIG. 5, the aggregated sub-devices is a JBOD (i.e. the data storagedevice is a JBOD, comprising a number of disks for storage 520, 530,580). The use of other types of storage sub-devices, such as flashcards, or other technologies, can be used in place of disks withoutdeparting from the scope and spirit of the disclosed subject matter.

There are embodiments wherein the arrangement by the disclosed subjectmatter of data addresses within a data address space are not necessarilymade adjacent, non-contiguous sequential, or non-adjacent; otherefficient arrangement of data addresses are possible. The arrangement ofthe data addresses within the data address space will in general becarried out to minimize the interaction with the data storage device. Insome embodiments, certain types of metadata relating to related dataobjects may be stored in either pre-determined locations, or inlocations that are specified according to a pre-determined heuristic ormethodology. While it may be possible to store metadata, or data capableof being used to calculate or determine metadata for a group of relateddata objects in a contiguous manner, to minimize storage of such data aswell as interactions required to send requests relating to such data, itis also possible to store such data at predictable or known storagelocations. Such data may located at every x^(th) data address, or atdata addresses which are calculated according to a hash function, orother function. In one embodiment, data requests are configured to beaccompanied by metadata that describes a characteristic of the data orthe data request (e.g. requestor's identity or IP address, time ofrequest, size of request, etc.) and then stored with a data address inthe data address space that may be contiguous and/or sequential with allthe accompanying metadata for similar requests (or requests for similardata types or data object types), but also at a pre-determined set ofaddresses that are determined according to a heuristic or function. Insome cases, the hash function to determine the data address may requirea key or other decoding information that is available from authorizedusers (or maintained by the system as long as power is maintained, so asto prevent theft or unauthorized removal of the device). While the aboveexamples regarding non-adjacent data addresses were described withrespect to the management of data addresses relating to stored metadata,non-adjacent data addresses that are arranged as described above mayalso be implemented for other data storage objectives, including thosedescribed elsewhere (e.g. to minimize read and write requests forlogically, spatially, and temporally related data objects or the datafrom such data objects). Such examples of additional storage objectivesmay include security or data-integrity, since the use of a hash functionto identify the locationally-related data addresses can utilizespecialized information (e.g. a private key). Without the key, a dataconsumer would not be able to call all the locationally-related datasince the pre-determined locations would not be determinable by such adata consumer.

There are disclosed herein systems and devices for organizing (includingsequencing, defragmenting, temporally associating, and view-basedarrangements) a data address space for a memory device, wherein thedevice address space comprises of addresses that correspond to physicallocation indicators, the physical location indicators corresponding withphysical locations on a storage medium within the memory device. Thecommunication may occur through an external bus.

With reference to FIG. 2, an exemplary data storage device 220 is shownin an operating environment 200, in which bus 230 is a PCIe bus. Thestorage device 220, in addition to the PCIe bus 230 for linking with thedata consumer 210 (which is a file system in this case), has a number ofadditional components. These include, but are not limited to, acontroller 240, an internal bus 250, one or more storage mediumcomponents 260, a translation layer module 270, and in some cases, acache memory 280. The controller 240 is a processor which processes andimplements instructions relating to the operation of the storage device.The internal bus 250 is the internal communication system that links thestorage media 260, the controller 240, the translation layer module 270,the cache 280 and the external bus 230; the internal bus 250, in somecases because it has a near-dedicated purpose and operation, isgenerally very fast and in some cases faster than the external bus 230.In addition to being dedicated, the internal bus 250 may in some caseshave an operational benefit or requirement to be faster than theexternal bus 230 so that internal communication and signal processingdoes not impact the overall speed and latency of the storage device 220itself. The internal bus 250 provides a communicative coupling betweenthe components of the storage device 220; in some cases, the coupling isbetween and amongst each of the storage medium components 260, thetranslation layer component 270, and the cache memory device 280; it mayalso include a coupling between those components as a unit and thecontroller 240, or alternatively, each of the components individuallywith one another, or different combinations thereof with differentcomponents forming a unit. The storage medium components 260 is/are thephysical medium upon which data is stored within the storage device 220;this may comprise volatile and non-volatile computer storage, such asDRAM, SRAM, SDRAM, as non-limiting examples of volatile memory, and ROM,PROM, EPROM, EEPROM, flash (including NOR, NAND, SONOS, phase-changeRAM, MRAM), or other examples of non-volatile memory known to personsskilled in the art of this disclosure. There may be one or a pluralityof discreet storage medium components 260; a single device may bescalably built with more or less storage capacity that meets the needsof the data consumer. The optional cache 280 provides a cache memory foroperations that occur within the device, such as may occur when data ismoved or copied to another cell or location on the physical memory. Suchcache memory may include RAM, such as SRAM, DRAM, SDRAM, and is in manycases high-speed and expensive (relative to the speed and cost of thestorage medium components). The cache memory may be volatile memory,embodiments with non-volatile cache are possible.

The translation layer constitutes a mapping between logical addresses,used by the data consumer to identify and locate specific data, andphysical addresses, that are associated with specific physical locationson the storage medium components. In some embodiments, the translationlayer constitutes, on the one hand, an address space of a finite (but insome cases variable) number of data addresses, each of which isassociated with a quantity of storage capacity; on the other hand, eachdata address is capable of being associated with a physical locationindicator that is indicative of the specific storage location on thestorage component. The translation layer may, in some embodiments, beembodied as a lookup table for associating logical data addresses withphysical storage locations.

A bus may be considered as a communication system that transfers databetween components inside a computer, or between computers; the use ofthe word bus in this context may be understood in some instances tocover all related hardware components (wire, optical fiber, etc.) andsoftware, including communication protocols or other protocols. PCIerelates to the Peripheral Component Interconnect Express), officiallyabbreviated as PCIe, and is a high-speed serial computer expansion busstandard designed to replace the older PCI, PCI-X, and AGP busstandards. These or other bus standards may be used in variouscomponents and elements disclosed herein without departing from thescope and nature of this disclosure.

In one embodiment, the subject matter of the instant disclosure can beimplemented to increase or maximize the amount of contiguous freeaddress space. A consequence of the increased amount of available andcontiguous address space for the storage resource (i.e. the flashdevice) is that the file system, which may be responsible for keepingtrack of where a data object is (or is to be stored) on the storageresource, can interact with the data of the data object with a single orreduced number of requests. If there is sufficient contiguous availabledata address space, the number of requests to, for example, write thatassociated data to the storage device is limited only by the maximumamount of data that can be associated with a data request, rather thanthe available contiguous address space. To the extent that the largestavailable contiguous data address space is less than such maximumrequest size, the data request must be split into a plurality of datarequests to each of the associated data address allocations, which wouldincrease the number of requests to be sent (including confirmations thatare sent in response to such requests) and also the number of writeoperations onto the physical memory space.

In some embodiments, addresses in the address space that are allocatedto data from a data object can be defragmented and/or sequentialized sothat data requests relating to that data object, such as read requests,can be minimized. In other words, the data object can be read with asingle data request; conversely, if the data is allocated acrossnon-sequential or fragmented data addresses, multiple read requests mustbe made.

In some embodiments, addresses in the address space that are allocatedfor data being temporally correlated (or data that is associated withtemporally correlated data objects) can be made sequential and/orgrouped together in the address space. As such that temporallycorrelated data can, for example, be “pre-fetched” to a higher tier ofdata, or otherwise prepared for a future interaction with all thecorrelated data in a minimal number of interactions. Since thetemporally correlated data has been made sequential or defragmented, theentire set of the correlated data objects can, for example, be retrievedin a minimal number of read requests; the amount of data returned fromany read request is limited only by the maximum amount of data that canbe carried within a response to such a request—rather than having tomake requests for each discreet sequential portion of data, which,depending on the level of granularity of each such sequential portion ofdata, can require at least equal number of requests but much more likelysignificantly higher number of interactions. Temporally related dataincludes data that is likely to be interacted with by a data consumer atthe same or close to the same time. Temporally related data may alsoinclude data that has similar levels of priority at similar times (i.e.data that is “cold” over the weekend may be grouped together in the dataaddress space and data that is “hot” at 9 AM on a Monday morning). Suchdata may be grouped together in a data address space, making prefetchingof all of the temporally related data occur with fewer interactions uponthe receipt of data requests for any one data or portion of dataassociated with the set of temporally related data.

In some embodiments, the implementation of management of address spacedisclosed herein can provide for an efficient manner of using views ofcharacteristics or properties of related data objects with minimalinteractions. As would be known to a person skilled in the art of theinstant disclosure, properties of related data objects can be accessedfor reasons other than the utility provided by the data objectsthemselves. For example, a file system may require frequent access toall the header information of all image or video data files that arestored therein. Currently, systems must either call all the related dataobjects, that is all the image or video files, or store such data in aspecific file elsewhere. The instant disclosure provides animplementation wherein the data associated with such metadata may beassociated with data addresses that are sequential and/or contiguousand/or locationally-related within the address space, therein providingfor more efficient interaction with this data (i.e. with minimal read orwrite requests) and without having to duplicate the data in additionaldata objects (which can become inconsistent with the true metadata) orhaving to retrieve all the data associated with the data objects.

Address space management in existing systems typically involves the dataconsumer calling the data, storing the data locally, then writing thedata to different addresses within the namespace. The instant disclosureutilizes the dedicated storage device processors/buses to re-allocatethe data to different physical data locations and thereby manage thedata address space for interaction with the data consumer, with reducedintervention from the data consumer.

As an example of the internal flash device infrastructure being utilizedto organize the available address space the FTL comprises of a mappingof logical addresses to physical storage locations in the flash. Thephysical storage spaces (referred to below as blocks) are locations inthe storage medium comprising of physical locations that must be erasedprior to being written; the medium further comprises of “erase blocks”which consists of multiple blocks that can only be erased concurrently.In examples herein, each row constitutes a discreet erase block.

In embodiments of the instantly disclosed subject matter, there areprovided methods of organizing the data address space of a data storagedevice. In one embodiment, there is provided a method for manipulating adata address space of a data storage device, the data storage devicecomprising at least one controller and at least one storage mediumcomponent, the data address space comprising a plurality of dataaddresses, each data address being associable with at least one storagelocation indicator that is indicative of a storage location in the atleast one storage medium component, the method comprising: selecting afirst data address in the data address space associated with a firstdata via a first storage location indicator, wherein the first datalacks a pre-determined storage-related characteristic with dataassociated with at least one data address that is locationally-relatedto the first data address; selecting a second data address from the dataaddress space, the second data address being available for addressingdata and being locationally-related to at least one data address that isassociated with a second data that shares the pre-determined storagecharacteristic with the first data; and generating an associationbetween the second data address and the storage location indicatorindicating the current storage location of the first data. In such anembodiment, a first data address which is currently included in the dataaddress space in an arbitrary, or otherwise inefficient, portionthereof, is identified for reallocation. For example, if the first dataaddress is fragmented (or otherwise separated from other in-use dataaddresses), available for addressing data, out of sequence with otherdata addresses that are related to the data to which the first dataaddress is currently addressing, or otherwise does not share astorage-related characteristic with adjacent or other data addressesthat have are locationally-related (e.g. by being adjacent or beinglocated at one of a predetermined data address locations or the dataaddress is identifiable according to a pre-determined function,methodology or heuristic), then the data address may be selected formanagement. The second data address is selected on the basis that: (1)it is or will be available for addressing the data currently associatedwith the first data address via the storage location indicator; and (2)it is or is capable of being locationally-related to another dataaddress that shares a storage characteristic with the first dataaddress. The storage characteristic may include whether the data addressis currently addressing: live data (defragmenting occupied dataaddresses) or non-live data (defragmenting available data addresses).The storage characteristics may also indicate two discrete pieces ofdata that are from the same data object, temporally related dataobjects, sequential in a data object or a data stream, part of the samemetadata or class of metadata for similar data objects or data objectsthat are part of the same class of data objects; such selection of themanagement of the data addresses for one or both of the addressesassociated with these pieces of data may occur under the methodsdescribed herein for the operational objectives discussed above (i.e.defragmenting, sequentializing, or otherwise renderinglocationally-related). Upon selecting first and second data addresses, astorage location indicator that is indicative of the current storagelocation is associated with the second data address; the updated storagelocation is either the same location as was associated with the firstaddress (in the case of a move wherein the physical storage locationindicator associated with another data address is changed to beassociated with the same storage location as the first data address iscurrently and, in some embodiments, the first data address is freed upor otherwise made available by deleting the association or tracking theavailability elsewhere in the data storage device and optionallynotifying the data consumer of the change in data address for thatdata), or it may be a new storage location indicator if the data wascopied and moved to a new storage location (optionally the data consumeris concurrently notified that the data address for that data haschanged).

In some embodiments, the first data address is disassociated with thestorage location indicator with which it was associated. This may bedone by deleting the association or by keeping track of which dataaddresses are not associated with live data in another lookup table inthe data storage device. In some embodiments, the first data addressneed not be disassociated as it may be possible and/or desirable forboth the first and second data addresses to point to the same storagelocation.

Locationally-related data addresses may, in some embodiments, includeadjacent data addresses. However, non-adjacent data addresses may belocationally-related in some embodiments by, for example, designatingspecific data addresses (which may or may not be adjacent) for certaindata or types of data. In such cases, the designated data addresses maybe according to a heuristic or regular pattern (e.g. every x^(th) dataaddress is for metadata for a given type of data object), or madeavailable in a lookup table. In other cases, the data addresses arelocationally-related because they are all identifiable according to apre-determined function or methodology, such as a hash function. Thiscan provide both ease of identification of the locationally-related dataaddresses but can also provide additional functionality, such assecurity, encryption, data verification, etc.

With reference to FIG. 3, an example will now be described, inaccordance with one embodiment, wherein a series of operations involvingdata requests to a memory device are carried out on a translation layermodule, in this case an FTL 310 that allocates 800 Gb worth of data, anda physical memory storage medium 350 that constitutes storage capacityfor 1.2 Tb of data.

Step 1: “000001f” is to be written to the storage device, and the filesystem sends a write request to data address “0”, which is currentlyavailable in the data address space in the FTL 310; the controllerdirects the write data to block 0.0 351A (which according to thisexemplary convention is row “0” and column “0”, or “R.C”) on thephysical memory storage medium 350 and updates the data address space315 in the FTL 310 such that the data address “0” 315A for this data isnow associated with the block “0.0”.

Step 2: Next, “000010f” is to be written, so the file system sends thewrite request to data address “1” 315B, which is also currentlyavailable in the address space; the controller directs the write data toblock 0.1 (i.e. row “0” and column “1”) 351B and updates the FTL 310 byassociating data address “1” 315B with “0.1” (i.e. the physical locationindicator for row “0” and column “1”).

Step 3: A request from the file system for the data stored at dataaddress “0” 315A (which is currently mapped to “0.0” 351A) is receivedto update that data and replace it with “000011f”; since the data inblock 0.1 351B must be preserved, but block 0.0 351A cannot be writtento until it has been erased—and only an entire erase block can beerased, which is in this example an entire row 351—the updated data isinstead written to block 1.0 352A (that is, row “1”, column “0”) and theFTL 310 is updated to associate the data address “0” 315A with “1.0”320A and the block 0.0 351A is identified as being null or available(this may occur in the FTL 310 as part of the same mapping table, orthere may be a separate component within the storage device to keeptrack of data that is null, or otherwise no longer needed, but not yeterased within the physical storage medium 350). As an aside, over timethe number of rows (i.e. erase blocks) having at least one block withlive data and at least one null data, will increase and as thisapproaches the maximum number of erase blocks, the controller will haveto do some housekeeping by transferring live data from a number ofdifferent erase blocks into a single erase block and update the FTL 310,as well as keeping track of the erase blocks that no longer have anylive data and either erasing them at that time or flagging them asavailable for erasing when data is to be written thereto.

Step 4: Next, “001000a” and “001000b”, which are data relating to, andtogether constituting, a single data object, are sent as part of a writerequest to data addresses “5” 315D and “3” 315F respectively; thecontroller stores these data at 2.1 353A and 2.0 353B respectively andthen updates the FTL 310 to associate “5” 315D with physical locationindicator 2.0 and “3” 315F with physical location indicator 2.1.Notably, these data are neither contiguous nor sequential (in the sensethat data from the beginning of the data object has a higher dataaddress, which may be important if data must be returned in a particularorder) in the data address space.

At this point in time, the address space comprises of non-contiguousavailable addresses and addresses that are non-sequentially orderedrelative to the order of the data in the stored data objects. Inaddition, the relationship between contiguous addresses (or indeed anytwo or more addresses) is completely arbitrary. Subject matter disclosedherein provides for an efficient manner of re-arranging the data addressspace that is used by data consumers using the constructs available inthe data storage device for such management, which reduces the overheadon the data consumer. As such, the data storage device may cause dataaddress “2” 315C to be associated with 2.0 (in the case of a move; inthe case of a copy, it may copy the data to another cell in the physicalmemory storage medium 350 and cause the data “2” 315C to be associatedwith that new cell) and then notify the data consumer of the newaddress. The same process would occur for data address “5”, whereby dataaddress “3” 325D would be associated with the data that was originallyassociated with “5” via the same storage location indicator (in the caseof a move) or another storage location indicator (in the case of copy).As a result, the largest contiguous block of address space goes from 1data address (each of 315C, 315E and 315G) to 3 data addresses (315E to315G inclusive).

In embodiments of the instantly disclosed subject matter, there aredisclosed methods for organizing the address space of a translationlayer component. In some embodiments, the organizing of address spaceinvolves changing the associations between logical addresses in theaddress space of the translation layer and the physical locationindicators in the translation layer. This can be accomplished in any ofa variety of ways including by assigning the physical location indicatorassociated with a first address to a second address in the addressspace, and then freeing the first address for further use. Optionally,the updated address for the data that is stored at the physical storagelocation is communicated to the data consumer (which may be, forexample, a file system) so that future interaction with that data issent to the appropriate address; this updating communication may happenat the time of reassignment, or when a data interaction with that dataoccurs, or by a bulk communication of all the data address reassignmentsthat have occurred during a previous time interval, or when the dataconsumer next requests data from the prior data address, or acombination of any or all of these. In other embodiments, the data iscopied from a first data location to a second data location, and asecond data address is amended so that it is associated with the seconddata location. In each case, changes to the data address space of thetranslation layer are effected. In some embodiments, the changes to thedata addresses within the data address space are implemented accordingto a “move” function, a “copy” function, or a “remap” function.

The “move” function or functionality for rearranging data addresses withthe data address space operates as follows. For a given first set ofarbitrary and unrelated data addresses in a data address space, whichare assigned in the translation layer module to data that is from thesame data object (or is otherwise related), a new group of dataaddresses in the translation layer module that are locationally-relatedare assigned the storage location indicators that currently apply to thefirst set of unrelated data addresses. The first set are then eitherreleased (although they may be maintained with the same physical storagelocation indicators for security or redundancy) and the data consumer isupdated with respect to the new addresses that should now be used tosend requests for the same data. It should be noted that there are otherunrelated processes that may relate to the data address, such asencryption, data verification or other processes that use, for example,a hash function and/or encryption key along with the specific currentdata address to carry out such processes; in such cases, the data devicewill update such processes (e.g. update the hash function informationbetween the data storage device and/or data consumer) so that the changein the address by simply moving the physical location indicator does notresult in errors when interacting with data.

The “copy” function or functionality for rearranging data addressescopies the data in one storage location and moves the copy to anotherstorage location and then uses an available data address in the dataaddress space to which the physical location indicator indicating thelocation of the new storage location is assigned. While the copyfunction utilizes some processing power within the data storage device,it minimizes the complexity relating to other peripheral processes (e.g.encryption, verification, etc. that may impact the move functionality asdescribed above). The new data address (or data addresses) are updatedwith the data consumer via a notification, at the time of copying or ata later time (e.g. a bulk notification of a number of copies or thenotification is provided upon the next data request for the data at theold address). The original or old address may be freed up, madeavailable, have its physical location indicator deleted for future usewithin the data address space. Alternatively, both copies may bemaintained for a number of reasons, including but not limited toredundancy, security, failsafe, or other reasons that would be known topersons skilled in the art. The copy functionality may, in embodiments,leverage the speed and dedicated functionality of the internal bus,over-provisioned memory, and existing protocols to manipulate the dataaddress space, rather than the data consumer requesting the data fromthe data addresses that it wishes to rearrange and then rewriting suchdata to the available data addresses within the data address space.

The “remap” function or functionality comprises both or either of themove or copy functionalities. In some cases of this function, whetherthe data storage device utilizes the move or copy functionality is notvisible or relevant to the data consumer and as such the remap functionpermits the data consumer to initiate data address space management buthow that rearrangement is performed is left to the data storage devicein any way it sees fit. The remap functionality may be implicit,including in cases where the data storage device presents a data addressspace in the form of data storage objects (e.g. OSD or object storagedevices, Seagate™'s Kinetic™ storage device, JBOKs, etc.). In the latterexample, when a number of storage objects have become sparse, unordered,or otherwise would benefit from a rearrangement of the data addressspace, the data storage device may be configured to recognize theconditions associated with the specific data consumer and its data forinitiating data address space management. In the case of data storageobjects, a number of sparse data objects may be combined into a reducednumber, possibly with the data contained within such objects arrangedinto a particular configuration that permits interaction with suchobject with minimized interaction (e.g. a reduced number of datarequests, such as read or write requests). In some cases, the datastorage device may initiate a remap upon the occurrence of one or morepre-defined conditions; as an example, in one embodiment the datastorage device self-initiates a remap when the largest contiguous blockof data addresses reaches a predetermined threshold and continues untilit reaches another predetermined threshold.

In some embodiments, there is provided an interface for managing a dataaddress space, including by manipulating or rearranging the dataaddresses within the data address space. The interface may be anapplication programming interface (“API”) that comprises a specificationor prescription for interaction between the data consumer and the datastorage device. The API may provide for “move”, “copy” or “remap”functions to permit the data consumer to initiate data address spacemanagement upon command. Alternatively, the API may permit the dataconsumer to provide the data storage device sufficient information toenable such data storage device to initiate data address spacemanagement on its own when certain conditions are determined to exist(e.g. sparseness or non-contiguousness of data addresses reaches apre-determined threshold). If a data address space characteristicreaches a pre-determined threshold, the data storage device may initiatedata address space management on its own. In some cases, the interfacemay comprise of a set of computer readable instructions on a mediumthat, when carried out by a computer processor, provide for theinterfaces between data consumer and data storage devices to manage adata address space.

In one embodiment, there is provided a data storage device interfacebetween a data consumer and a data storage device, the data storagedevice comprising (i) at least one storage medium component with storagelocations and (ii) a data address space comprising a plurality of dataaddresses, the data storage device interface comprising: a functionalitycommand for providing a data address reallocation mode for manipulatingdata addresses in the data address space; a source data addressidentifier that identifies a first data address for reallocation, thefirst data address in use for addressing a data and being associatedwith one storage location indicator indicative of the storage locationstoring the data; and a destination data address identifier thatidentifies a second data address, the second data address beingavailable for addressing the data; wherein the functionality commandinstructs the data storage device to allocate the storage locationindicator indicating the current storage location of the data. In suchan embodiment, the data storage device interface is an API oralternatively a computer readable medium having instructionscorresponding to the API stored thereon. The functionality command may,in some embodiments, correspond to the move, copy or remapfunctionalities described herein, although other functionalities forcausing the data storage device to manipulate the data address space maybe used. Also in such an embodiment, the API may cause allocation of thestorage location indicator of the data, currently in association withthe first data address, to the second data address. Also within thescope of this embodiment, the API may cause the data stored at thestorage location currently associated with the first data address to becopied to another storage location and then associate the storagelocation indicator, which indicates the location of that second storagelocation, to the second data address. Further, the functionality commandmay utilize a remap functionality, corresponding to the remapfunctionality described above.

In some embodiments, there are provided data storage devices with aplurality of controllers, each of which may communicate with oneanother, and each of which may have dedicated data storage resourcesand/or translation layer modules. The data consumer interacting withsuch data storage devices may be presented with a data address spacethat accesses or uses physical storage resources across all suchcontrollers; in some of these cases, the controllers themselves may beresponsible for assigning which controller is responsible for whichportion or aspect of the data address space. In some such cases, thedata consumer would have no visibility or transparency to whichcontroller utilizes which portion of the data address space. Forexample, in the case of a data storage device that utilizes data storageobjects, the interface may simply provide access to data storageobjects, each of which exist on storage that is controlled by differentcontrollers; the rearrangement of such data address space by, forexample, placing the sparse data sets across multiple data storageobjects into a reduced number of data storage objects, may in some casesnecessarily require that controllers within the data storage devicecooperate to remap data from a data storage object that is part of thedata address space assigned to one set of data resources to another setthat is assigned to a different controller; all of this may not bevisible to, or require participation from the data consumer (except insome embodiments that request information regarding the conditions uponwhich initiation of the data address space management by the datastorage device would occur).

In embodiments, the changes to the data address space of the translationlayer may be coordinated to result in one or more outcomes that impactoperational characteristics relating to improved efficiency of the datainteraction and use by the data consumer, or for the storage deviceitself, or both. These outcomes include defragmentation of thecontiguous available or unavailable data addresses, sequentializing dataaddresses such that the order of the data addresses for specific datamatches the order of that data in the data object of which the data is aconstituent part, grouping data addresses together that are used fordata that is frequently accessed or is likely to be accessed at the sametime or otherwise have a temporal relationship, grouping data addressesthat are used for metadata or other data that may relate to the state orother characteristics of multiple data objects is accessed. The lattermay occur when a view of a specific subset of metadata relating to alldata belonging to a class of data; as a non-limiting example, a filesystem may wish to assess characteristics of all stored files of aparticular type, such as determining how many media files with aspecific extension exist in storage and/or how many or which of themhave been read or altered in a previous time interval. Prior to suchdata being accessed, or if such data is access frequently, embodimentsof the instantly disclosed subject matter can group together the dataaddresses (and optionally in a sequence related to the sequence of theunderlying data, if the order of such data is associated with the use ofthat data by the data consumer) so that such metadata need not be copiedto a separate location, and a minimized number of requests (either readsor writes) relating to such data is needed to interact with that data.Indeed, by facilitating (i) sufficient contiguous available data addressspace, interaction with the data storage device is minimized for writingdata to the data storage device; and/or (ii) by sequentializing dataaddresses in accordance with some relationship between the underlyingstored data (that is being addressed), the interactions for reading suchdata is minimized and/or (iii) by placing the metadata, ormetadata-related data in identifiable or predetermined locations. Therequests are not impacted by, for example, the size of the availableaddress space or the size of the chunks of the data objects; thelimiting factor shifts towards the maximum size of the amount of datathat can relate to a given data request.

In embodiments that comprise spinning disks as the storage mediumcomponent, there are generally one or more magnetic spinning disks withan armature apparatus positioned and moveable so that the head of thearmature can apply a magnetic field to the disk, thereby changing thestate of the disk at that location. The disks are generally spinning sothat tracks of sequential memory locations are located in concentricrings around each spinning disk. Data may be stored anywhere on therespective spinning disks, so seek time to get to read or write data ona disk is a function of how many tracks are associated with the data andhow far the armature needs to move between tracks. Because of thissequential reads/writes on a single track are generally faster (e.g. 100MB/s) than random reads/writes (e.g. 2 MB/s). A translation layer may beused to provide data addresses for data stored on the disks to keeptrack of which tracks are used for storing that data, and where amongstsaid tracks the data is specifically stored. Using a translation layerto maintain related data, for example, on the same or adjacent tracksmay be facilitated by the use of having a data address space to trackphysical locations within a spinning disk medium.

There are a number of functionalities that are coded in the storagedevice, generally firmware controllers with various protocols (e.g. SASor Serial Attached SCSi, SATA or Serial ATA). These controllers maycomprise of a CPU, a memory and a high-speed communications bus;alternatively, the controllers may operate in conjunction with memoryand a bus located elsewhere in the data storage device. The protocolsconsist of a number of commands, and may include seek, read, write,move, or copy. Many storage devices have a communications bus with filesystems, or other application level entities, that is around 600 MB/s.This is because, even with multiple spinning disks within a givenstorage device, or even groups of storage devices that work concurrentlyin presenting a single virtual storage unit, the physical limitations ofthe layout of the data on spinning disks will in general never exceedthis upper limit of communication speed.

In contrast, flash memory does not have similar problems relating to thephysical layout of the data. Flash does, however, sometimes suffer fromanalogous logical issues. In many flash memory systems, the storagemedia is different from spinning disks in that storage locations must beerased before they are written upon. That is, you can read from a flashstorage location, you can write to it (by setting a 1 to a 0) or you canerase it (by setting a 0 to a 1). In many cases, flash memory is noterased at the same level as it is written. That is to say that whileindividual flash memory units can be written upon, erasing generallyoccurs to groups of memory units; so in order to write something new ona group of flash memory units (such as memory cells), the entire erasegroup must be erased before doing so. In cases where some data in theerase group must be maintained, the entire group must be moved or copiedelsewhere. In order to manage this problem, flash memory devices haveused “Flash Translation Layers”, or FTL, which operates as a mapping ofphysical address spaces on the flash media. If a group data needs to beerased in order to write something new, the existing data that must bemaintained is copied elsewhere and the FTL maintains a mapping of wherethat moved data has been moved to. When all erase groups have at leastone cell of live data, the system must re-arrange the data by puttinglive data into existing erase blocks. The flash memory device, duringthese times, has significantly reduced performance (throughput goesdown, and latency goes up). This leads to periodic but short-lived timeperiods of significant reduction in performance. Flash memory deviceshave mitigated this problem by over-providing a significant excess ofstorage and processing capabilities to avoid this hampering performance.In some products, for example, an 800 GB flash card actually comprises1.2 TB of storage. In other products, the over-provisioned storage ismade selectively available for storage, thereby reducing performance, inother products the additional storage capacity is used exclusively formanaging the block-erase problem. This has led to commercial flashmemory devices being manufactured and distributed with an excess amountof storage media (e.g. multiple NAND flash devices) with a separatehigh-speed FTL media, a dedicated controller, and high-speed anddedicated internal bus for communication therebetween. In fact, theinternal bus is generally much faster than the external PCIe or SASbuses (generally, 32 GB/s and 600 MB/s, respectively). This dedicatedand high speed sub-set of functionality of flash memory devices canavoid the “re-arrangement” problem and provide uninterrupted performancelevels of 200000 b/s of sequential reads, 80000 b/s of random reads and50000 b/s of random writes. Many different types of commodity memorystorage devices have internal mechanisms to map device address spaces tophysical locations on a storage medium, including but not limited toflash memory devices and spinning disk devices.

Flash memory devices have FTL tables and additional storage to handlethis problem. The FTL tables consist of a mapping of device addressspace that maps to a device physical address space. The reason for thisis that when a cell in flash needs to be written to, an entire blockmust be deleted, which is not possible if there are other cells in thatblock that contain live data. In order to overcome this, the FTLutilizes its address space for mapping the data to the each physicalcell and then changes that to a cell in a free erase block;

Step 1: write(“hi”,0)—>“hi” written to cell (0,0), which is associatedwith “0” in the FTL

Step 2: write(“foo”,1)—>“foo” written to cell (0,1), which is associatedwith “1” in the FTL

Step 3: write(“bye”,0)—>“0” cannot be written without erasing both (0,0)and (0,1) because they are in the same erase block, so “0” is associatedin the FTL with (1,0), and (0,0) becomes unassociated with any data andcan be deleted later (there may be a separate mechanism within datastorage systems to account for and keep track of whether cells areassociated with “live” data that is current and not an older versionthat exists as part of an erase block which could not be erased and forwhich an newer version exists elsewhere as mapped by the FTL).

Over time, the address space becomes fragmented, or randomly, orarbitrarily ordered, which forces a file system that is associating dataobjects with data addresses in memory to place them, or read from them,for example, in ways that are not optimal for memory performance. Forexample, to write a data object to storage wherein the data addressspace is fragmented into portions such that there are not enoughsequential or contiguous data addresses for all the data of the dataobject, separate write requests to each applicable available dataaddress must be transacted. In cases where there is a single or smallnumber of contiguous and available data addresses available for all ofthe data of the data object, then minimal number of write requests arerequired to write data to storage.

Currently, the sub-optimal placement of data within an address space mayrequire that the file system (or other applicable data consumer) (1)read the data associated a data address that it wishes to reallocate bytransmitting and receiving a response from a read request, (2) storethat data in cache (or other local memory), and then (3) transmit andreceive a response from a write request to data storage device with thedesired address space. This requires the use of the PCIe bus, orexternal bus, which may be in general slower than the internal bus.Moreover, it consumes file system processing time, network bandwidth,and increases latency. Lastly, in current systems, unless the filesystem implements these actions proactively, the data will be stored insub-optimal locations resulting in an increase in data operations inreading and writing on the data storage device. This disclosuredescribes a storage-related interface comprising a translation layer,associating logical data addresses and device address space mappings (orphysical storage location indicators), combined with control elementsthat organizes data placement, to optimize data address allocation tominimize the number of operations required to interact with that data.

In one embodiment, data from a data object, e.g. a file, are associatedin one or more portions with respective data addresses. Since there arelimited data addresses available, the data portions must be associatedwith the available addresses in the address space; in the case of awrite, if the available data addresses would not accommodate the entiredata object, because for example large portions of the data addressspace comprise non-contiguous data addresses, then significantly morewrite requests must be made to write the portions into the multipleavailable spaces; if, on the other hand, the FTL/flash controller isconfigured to move/copy/organize data within the FTL/physical addresseswithout additional data transactions by the file system, the availableaddress space can be organized to ensure that the data addresses thereincan be organized to increase their effectiveness, by for example,increasing the contiguousness of the available address space bydefragmenting the address space.

Example

With reference to FIG. 4, the same data address space is provided at 4different points in time 410, 420, 430, 440, the data address space loghaving a plurality of segments, the first three of which shown as 401,402, 403. The data address space supports the following system: anenterprise storage system comprising multiple data storage devices basedon PCIe Flash, in this case Intel 910s as a base. The enterprise storagedevice comprises a high-performance, high-scale product, which unlikesingle-box filers, integrates with SDN 10 GB switches to provide linearscale and capacity. The system has been validated against a pool of 20Intel 910s, served over 20 10 GB ports to a replicated, fully-random,80/20 fully replicated 4K NFS performance of almost 2M IOPs. The NFSfile system constitutes the data consumer in this example. Thisenterprise product comprises a deep software stack, spanning flash, OS,network protocol design, switch chipset integration, which may befocused on performance and scale for large enterprise storageenvironments. The individual storage nodes receive read and writerequests from many clients at once; the goal is to minimize IOPs andmaximize performance.

In FIG. 4, the data address space at t₀ 410 shows the data address spaceprior to use. When the data address space is at t₁ 420, the requeststream may use a log-structured on-disk layout where merges and appendsare written in large, aligned blocks 421 at the data address space tail422; data address space entries are self-describing, with no dependentmetadata updates. In-memory mapping state allows reads to be serveddirectly from the data address space.

At t₂, the data address space 430 has become fragmented: data has beenoverwritten and files are deleted, resulting in “dead data” spaces 431A,431B, and 431C. Contiguous chunks of logical block addresses (“LBA”) inthe data address space would be preferable for the data address space,in order to avoid breaking up data requests to accommodate data storagefor data objects that are larger than the largest available contiguousgroup of LBA. The LBA may be virtualized through an indirection table,but this would mean fragmenting data address space, and then appendingrequests and amplifying IOPs, as well as additional overhead by the NFScomputing device. Prior solutions required that a segment based “garbagecollector” that cleans from the head of the data address space, whichcleans a region of multiple megabytes, as shown in the data addressspace after a third time interval 440. In so doing the NFS computingdevice reclaims dead data addresses, and moves live data 421B to thetail of the data address space 441, in order to free LBA regions 442, bysending data read requests, storing locally, and then sending data writerequests to the free tail of the data address space 442. Moving data inthis manner, however, is expensive.

In some alternative versions of this exemplary system, there is a hybridlog/heap, to allow for long-lived data and with independent treatmentfor cold and hot data to quickly reclaim data that is frequentlyoverwritten. Even with these optimizations, under high write loads thathave even small amounts of long-lived data, segment cleaning is thedominant overhead and leads to 60-70% performance fall-off, and R/Wdependencies in coping from one part of the data address space toanother add additional latencies. Given that the FTL of flash memory,comprising a flat indirection table, is already serving this role,implementation of the subject matter disclosed herein provides highlyefficient management of the data address space, without the highoverhead and cost of requiring the NFS computing device to manage thedata address space as described above.

Below are two exemplary implementations of an aspect of the systemdescribed above. Since the NFS cannot ask for mappings from the lowerlayer (i.e. the physical addresses of R/W blocks within the flashmedia), there is provided an interface comprising the followingcommands:

move(src_lba, dst_lba, size) or remap(src_lba, dst_lba, size)

This command moves or remaps data on flash from a source LBA (src_lba)to a destination LBA (dst_lba). Size is a multiple of flash block size,when addresses are block aligned. The call can fail if alignment or sizeis invalid. Remap may result in a move command, but can also result in acopy depending on settings. The commands in either case are implementedas an update to the flat indirection table (e.g. the flash translationlayer in a flash device). While simple in implementation, some systemsmay result in some additional complexity using a “move” based command.In particular, from an FTL perspective, an accounting for what happensto data at the original mapping must be contemplated. Mapping the samedata twice likely requires some sort of reference counting. Also, analternate approach would replace the src_lba with a zero page. However,this raises concurrency issues in some stacks, with regard to in-flightreads, which may refer to src_lba.

In an alternative implementation, instead of moving location indicatorsbetween LBA, the following copy command leverages existing FTLfunctionalities:

soft_copy(src_lba, dst_lba, size)

This command may share some of the same characteristics as move( ) butthe interface leaves valid copies of the data at both addresses. The FTLcan implement either by remapping or by internally copying the data.This eases application development slightly, because there is noconflict with in-flight reads. In some embodiments, this command mayimplement the following: (a) generate a soft_copy of dst_lba, (b) updatemetadata, including at the NFS, and then (c) reclaim src_lba.

Referring to FIG. 6, there is shown a flowchart that is representativeof a method 600 in accordance with one embodiment of the instantlydisclosed subject matter. Beginning at step 610, the controller of thedata storage device initiates data address space management; thecontroller may receive a command from a data consumer, or otherintermediary that has access to the data request path between the dataconsumer and the data storage device, wherein such commands may includecopy, move, or remap, or another similar command that initiatesreallocation of data addresses in the data address space. In some cases,the controller may not receive an explicit command for everyreallocation of a given data address or set of data addresses, butrather detects certain characteristics that may be receivedindependently from a data consumer (or other intermediary) or as part ofthe data in a data request transaction. If received as part of the datarequest transaction, such data may be a discrete portion of such datarequest or it may be identified and assessed by the controller (by, forexample, analyzing the payload or the header information of a packet orother protocol data unit). Upon detecting the existence of certaincharacteristics, or that some criteria or threshold of suchcharacteristics has been reached, the controller may initiate dataaddress space management; else it may stop at this point 680. If dataaddress space management is initiated, the controller, in step 620,seeks to identify a first data address that is currently being used by adata consumer to address data stored in the data storage media of thedata storage device (and which is associated with a physical locationtherein by a physical location indicator in the translation layer). Suchidentification may be an explicit identification provided by the dataconsumer (or intermediary) in a move, copy or remap command, or thecontroller may have functionality stored thereon to identify dataaddresses that should or can be reallocated to achieve one or moreoperational objectives (e.g. grouping data addresses that are used foraddressing related data) particularly when data address managementoccurs based on the automated detection of characteristics of the data,data requests, or data address space. It should be noted that in somecases, the data address may be allocated for data that is not yet storedin the data storage medium; for example, a data address, and a physicallocation by association in the translation layer, may both be associatedwith one another and allocated even if there is no data or live datacurrently stored at that physical location in the case of data writerequest that is or is potentially expected to be received. Similarly, atstep 630, the controller identifies a second data address that isavailable for the data associated with the first data address; again,this may be explicitly presented as part of a command or othercommunication from the data consumer (or other intermediary), or it maybe determined by the controller with or without additional informationfrom the data consumer. At step 640, the controller determines whether amove or a copy is preferable; this may be an explicit requirementprovided by the data consumer, and it may be necessary depending onother background requirements of the data storage device to the extentthat there are any data address-dependent functionalities associatedwith the data storage. For example, there may be some encryption thatutilizes information from the data address in a hash function or otherencryption methodology that is utilized to encrypt and decrypt the data;in such cases, this overhead must be handled by amending the hashfunction or related information after the data address associated withthat data has changed, including by updating the hash values or theassociated keys to decrypt according to the source location andre-encrypting based on the new location. While this may include someadditional overhead, including by passing the information through theencryption function (e.g. API), this can be located within thedevice-internal functionalities, while therefore avoiding the higherand/or different (e.g. networking vs. processing) overheads that arerequired under current systems to manage data address space and theunderlying locations on the physical media.

Similarly, there may be support for other background functionalitiesthat may have some data address dependent component, including but notlimited to, storing back up or redundant copies (a determination of whatis live or current data must be made or maintained elsewhere), ensuringdata integrity (e.g. TCP data integrity checks during data requesttransactions), and any other data address dependent functionality. Tothe extent that such data address dependent functionality is present,and there is no overhead handling of such functionality upon the move ofa data address, the controller must select to initiate a copy at step670. If there is overhead handling, then the controller may selecteither a copy 670 or it may select a move at step 650, depending onresource requirements and availability. At step 660, the controllerhandles the address-dependent overhead. In some embodiments,particularly in (but not limited to) cases where the controller hasinitiated data address space management automatically upon detection ofa characteristic or criteria thereof (e.g. reaching a threshold), thisprocess may be iterative until such characteristic is no longer presentor is within certain operational limits. If so, the process returns tostep 610 where the controller assesses whether data address spacemanagement should continue or stop 680.

In some embodiments, the subject matter relates to the management of thedata address space, as well as the associated physical data storagelocations, in data storage resources using (i) characteristics betweendata in the physical storage locations and (ii) the device-internaltranslation layer management schemes implemented by modern storagedevices, including but not limited to flash devices and othernon-volatile data storage devices (e.g. PCiE devices), spinning disks,and CPU/RAM arrangements in general purpose computers.

In general, embodiments hereof facilitate: (1) the offloading of datatransfer between physical locations on the data storage device todevice-internal bus and transfer compute resources; and/or (2) allowingexplicit manipulation of translation layer by off-device applications;and/or (3) the use of data-specific characteristics to more optimallymanage either or both of the data address space and storage locationwithout the associated overhead of sending and receiving data requestsbetween data consumer and device.

Encryption: stored information is encrypted using the address as anargument to the encryption function; in order to move a data fromlocation to another, or indeed re-mapped, it must first be passedthrough the encryption function to decrypt using the first address andthen encrypted using the new address to ensure that it remainsconsistent.

For example, there may be encryption of data on flash or other devicesthat utilizes information from the data address in a hash function orother encryption methodology to encrypt and decrypt the data; in suchcases, this overhead must be handled by amending the hash function orrelated information after the data address associated with that data haschanged, including by updating the hash values or the associated keys todecrypt according to the source location and re-encrypting based on thenew location. While this may include some additional overhead, includingby passing the information through the encryption function (e.g. API)when moving or copying data as part of the data address space orlocation management methods, this can be carried out within thedevice-internal functionalities. This therefore still avoids the higherand/or additional (e.g. networking) overheads that are required undercurrent systems to manage data address space and the underlyinglocations on the physical media.

In general, more and arbitrary processing can by pushed onto thedevice-internal bus and transfer compute resources. For example,transcoding, indexing, journaling of bulk operations, can be pushed downonto the device-internal bus and transfer compute resources.

By exposing control of the data address space, as well as the storagelocation management, to the data consumer, the data consumer can utilizeinformation relating to the data (e.g. data characteristics or metadata)to determine optimal or more efficient placement of data within flashmemory. For example, the data consumer is aware of or has access toinformation that characterizes data. This data is either unavailable toor unusable by the translation layer and/or the device-internalcomponents when managing or placing data addresses and/or storagelocations. By exposing the control that is available to the dataconsumer, however, more optimal data placement on the data storagemedium and selection of the data addresses can be implemented. Forexample, the data consumer can be configured to ensure that data ofsimilar priority is placed within the same erase blocks. As such, theefficiency and timing of erasing blocks can be improved. Erase blockswill not require reconciliation (e.g. garbage collection to free upstale or dead but unerased portions of the erase block) for a period oftime at least as long as the first data unit to be updated; if cold andhot data can be arranged onto the same respective erase blocks, forexample, “cold” erase blocks will likely not need to be reconciledfrequently and only the “hot” erase blocks will require frequentreconciliation (unlike current systems where the relationship betweendata and the erase block is not manageable in this way and as such thelikelihood of any given erase block requiring frequency is related tothe likelihood of receiving more “hot” data, which is arbitrary andessentially the same across all erase blocks). For example, low prioritydata (i.e. cold) which are accessed and/or updated infrequently can becollocated on common erase blocks; such erase blocks will be much lesslikely to incrementally increase the number of stale blocks. Conversely,higher priority data (i.e. hot data) can be collocated on common eraseblocks; such erase blocks will have a greater number of data unitsthereon become stale and the erase block can be garbage collected soonerin time with fewer live blocks having to be relocated and managed. Thissaves overall processing resources, reduces the number of times thatflash memory has to be garbage collected, and may increase the lifetimeoverall of the flash memory device. As FTL implementations are generallyunaware of data characteristics (or unable to manage data placement inaccordance therewith in any event), an arbitrary mixing of high and lowpriority data within erase blocks will lead to inefficient dataplacement, wherein erase blocks have to be garbage collected and withoutregard to whether certain blocks thereon may be high priority andunlikely to change and whether a small number or a large number arelikely to change frequently (and thus not being a good candidate forgarbage collection). The data consumer may be configured to assesswhether an erase block, given the priority of the data stored thereon,is a good candidate to move the data thereon as part of garbagecollection, or if the erase block will, given more time, become morestale in an upcoming time interval, at which time the garbage collectionwill only require moving a smaller number of data blocks from such eraseblock.

In some embodiments, there may be implemented “Transactional Memory”applications; such applications include API's to implement bulk ormulti-write transactions directly by the device-internal bus andtransfer compute resources, and avoid the requirements forapplication-based journaling or other ways of managing memorytransactions. Transactional memory applications often involve the use ofcomplex algorithms, as well as expensive processing overhead, to ensurethat data transactions are ordered and in some cases journaled orindexed to ensure the integrity of transactions. For example, to avoid aloss of integrity of data which may occur if there is a componentfailure during a data transaction, many systems include certain failsafeprecautionary measures. This might include, but is not limited to,ensuring that data is copied prior to deleting the source data whenmoving data; it may also include journaling all steps in datatransactions so that the transactions can be recorded and re-implementedin case of system failure. Such transactional safeguards and overheadare often called “transactional memory.” In accordance with embodimentshereof, the overhead of such transactional memory can be pushed down tothe device-internal translation layers and components.

Similarly, more and arbitrary processing can be pushed down onto thedevice-internal bus and transfer compute resources. For example,transcoding, indexing, journaling of bulk operations, can be pushed downonto the device-internal bus and transfer compute resources. Wheretranscoding may be utilized, for example in the case of generatingmultiple versions of media for download and playback on differentlyconfigured devices and/or which may have access to different networkresources, the device-internal transfer and compute resources may beused to provide a subset of the data corresponding to a differentlytranscoded version. As such, the data consumer need not re-createmultiple versions, but rather simply request the same data with adifferent transcoding and permit the device-internal bus and transfercompute resources to return only the subset of data corresponding to therequested transcoding (e.g. for a video stream, only 2 frames per secondor fewer pixels per frame). Similarly, indexing and journaling may becarried out by the device-internal components, translation layer and thedata storage medium.

In some embodiment, there is provided data storage system for managingdata storage from a data consumer, and related methods and devices,wherein the data storage system comprises: a storage medium defining aplurality of storage locations, each one of which having a uniquestorage location indicator associated therewith; a translation layerdefining a data address space having a plurality of data addresses, eachone of which being assignable to at least one said unique storagelocation indicator; a resident controller carrying out computer readableinstructions for writing and reading data in said storage locations andmanaging data addresses assigned thereto in said translation layer, saidresident controller being further operable to carry out storage locationoptimization instructions in reassigning at least some of said storagelocations to optimize access and use of said storage medium via saiddata address space; and an external communication interface accessibleto the data consumer to service data requests received therefrom to saidstorage medium via said data address space. Such embodiments, whichprovide for an ability to optimize storage locations instead of, orconcurrently with, the data address space, expose functionality to thedata consumer to, for example, designate portions of the storagelocations for specific data, specific types of data, and/or data sharingone or more characteristics. The storage location optimization may beimplemented by providing information to the data consumer relating tothe storage location associated with a specific data address, andtherefore only access by the data consumer to the data address space isrequired; in other embodiments, the resident controller may provideaccess or information to both the data address space and the storagelocations. Subject matter described herein to move or copy data tomodify applicable data addresses, can therefore, by implemented for theobjective of modifying applicable storage locations.

While the present disclosure describes various exemplary embodiments,the disclosure is not so limited. To the contrary, the disclosure isintended to cover various modifications and equivalent arrangementsincluded within the general scope of the present disclosure.

What is claimed is:
 1. A data storage system for storing data from adata consumer, the data storage system comprising: an externalcommunication interface with the data consumer; a storage mediumcomponent comprising a plurality of storage locations, each of thestorage locations having a unique storage location indicator associatedtherewith; a translation layer module defining a data address spacehaving a plurality of data addresses, each of the data addresses beingassociable with at least one said unique storage location indicator; anda controller configured to carry out computer readable instructions forwriting and reading data in the storage locations and amendingassociations in the translation layer module between data addresses andthe storage location indicators; wherein the data address space isaccessible by the data consumer via the external communication interfacefor addressing data requests to the storage medium component and whereinthe controller is configured to manipulate arrangement of the dataaddresses in the data address space by changing the associations betweenthe data addresses and the storage location indicators.
 2. The dataaddress management system of claim 1, wherein the controller manipulatesthe data addresses in accordance with one or more storage-relatedoperational objectives.
 3. The data address management system of claim2, wherein the one or more storage-related operational objectivescomprise at least one of: defragmenting the data address space,sequentializing the data addresses in an order corresponding to an orderof the associated data in a data object, arranging the data addressspace by associating temporally related data with locationally-relateddata addresses, arranging the data address space by associating datacomprising related metadata with locationally-related data addresses. 4.The data address management system of claim 1 wherein the controllermanipulates the data addresses by making a copy of given data associatedwith a first data address, writing the copy to a second storagelocation, and associating a second data address in the translation layermodule with a given storage location indicator related to the secondstorage location.
 5. The data address management system of claim 4,wherein the system is further configured to communicate a notificationto the data consumer that the second data address is associated with thegiven data upon at least one of the writing of the copy to the secondstorage location, a notification threshold being reached wherein aplurality of notifications are generated by the controller, and receiptof a subsequent request by the data consumer for the given data.
 6. Thedata address management system of claim 1, wherein the controllermanipulates the data addresses by associating a second data address witha same storage location indicator currently associated with a first dataaddress.
 7. The data address management system of claim 6, wherein thesystem is further configured to communicate a notification to the dataconsumer that the second data address is associated with given datalocatable via said same storage location indicator upon at least one of:the associating of the second data address with the same storagelocation, a notification threshold being reached wherein a plurality ofnotifications are generated by the controller, and receipt of asubsequent request by the data consumer for the given data at the firstdata address.
 8. The data address management system of claim 6, whereinthe controller subsequently disassociates the same storage locationindicator with the first data address.
 9. The data address managementsystem of claim 1, wherein the data storage device comprises one or morememory storage types from the following: flash memory, spinning diskmemory, and a combination thereof.
 10. The data address managementsystem of claim 1, wherein the controller is further configured toarrange data addresses in the data address space upon one of thefollowing: a pre-determined scheduled time, a pre-determined storagecondition occurring, a command received by the data consumer, and acombination thereof.
 11. The data address space management system ofclaim 1 wherein the data address space is one of the following:consecutive data addresses, a plurality of subsets of data addresses, aplurality of configurable data objects, and a combination thereof.
 12. Amethod for manipulating a data address space of a data storage device,the data storage device comprising at least one controller and at leastone storage medium component, the data address space comprising aplurality of data addresses, each data address being associable with atleast one storage location indicator that is indicative of a storagelocation in the at least one storage medium component, the methodcomprising: selecting a first data address in the data address spaceassociated with a first data via a first storage location indicator,wherein the first data lacks a pre-determined storage-relatedcharacteristic with data associated with at least one data address thatis locationally-related to the first data address; selecting a seconddata address from the data address space, the second data address beingavailable for addressing data and being locationally-related to at leastone data address that is associated with a second data that shares thepre-determined storage characteristic with the first data; andgenerating an association between the second data address and thestorage location indicator indicating the current storage location ofthe first data.
 13. The method of claim 12 further comprising:disassociating the first data address and the first storage locationindicator.
 14. The method of claim 12 further comprising copying thefirst data to another storage location.
 15. The method of claim 12,wherein the pre-determined storage characteristic comprises at least oneof the first and second data comprise live data, the first and seconddata comprise dead data, the first and second data are both part of adata object, the first data and second data are sequential in the dataobject, the first data and the second data are temporally related, thefirst data and second data are related metadata, and the first data andsecond data comprise of data used to determine related metadata.
 16. Themethod of claim 12, wherein locationally-related data addresses areadjacent data addresses.
 17. The method of claim 12, wherein thelocationally-related data addresses are data addresses located atpredetermined locations within the data address space.
 18. The method ofclaim 17, wherein the predetermined locations are identified by at leastone of a function, a hash, and a lookup table.
 19. The method of claim12, wherein the data storage device comprises at least one of a flashmemory and a spinning disk memory.
 20. A use of the system of claim 1 tominimize interaction with the data storage device by the data consumerby manipulating the data address space to arrange data addresses in thedata address space for at least one operational objective.
 21. The useof claim 20, wherein the at least one operational objective isdefragmenting data addresses to minimize write requests required towrite additional data objects to the data storage device.
 22. The use ofclaim 20, wherein the at least one operational objective issequentializing data addresses relating to data from asequence-dependent data object to minimize read requests required forthe sequence-dependent data object.
 23. The use of claim 20, wherein theat least one operational objective is to group data addresses associatedwith temporally-related data to minimize read requests for thetemporally-related data.
 25. The use of claim 20, wherein the at leastone operational objective is to group data addresses associated withsimilar metadata from multiple related data objects to minimize requestsrelating to an assessment of at least one characteristic of at least oneof the multiple related data objects.
 26. A data storage deviceinterface between a data consumer and a data storage device, the datastorage device comprising (i) at least one storage medium component withstorage locations and (ii) a data address space comprising a pluralityof data addresses, the data storage device interface comprising: afunctionality command for providing a data address reallocation mode formanipulating data addresses in the data address space; a source dataaddress identifier that identifies a first data address forreallocation, the first data address in use for addressing a first dataand being associated with a first storage location indicator indicativeof the storage location storing the first data; and a destination dataaddress identifier that identifies a second data address, the seconddata address being available for addressing the data; wherein thefunctionality command instructs the data storage device to allocate thefirst storage location indicator to the destination data address.
 27. Adata storage system for managing data storage from a data consumer, thedata storage system comprising: a storage medium defining a plurality ofstorage locations, each one of which having a unique storage locationindicator associated therewith; a translation layer defining a dataaddress space having a plurality of data addresses, each one of whichbeing assignable to at least one said unique storage location indicator;a resident controller carrying out computer readable instructions forwriting and reading data in said storage locations and managing dataaddresses assigned thereto in said translation layer, said residentcontroller being further operable to carry out address spaceoptimization instructions in reassigning at least some of said dataaddresses in said data address space to optimize access to said storagemedium via said data address space; and an external communicationinterface accessible to the data consumer to service data requestsreceived therefrom to said storage medium via said data address space.28. The data storage system of claim 27, wherein said residentcontroller is operable to receive said address space optimizationinstructions from the data consumer via said external communicationinterface.
 29. The data storage system of claim 27, wherein saidresident controller is operable to implement self-diagnosticinstructions in evaluating address space optimization and internallyoutput said optimization instructions as a function thereof.
 30. Thedata storage system of claim 27, wherein said optimization instructionscomprise grouping instructions that, when implemented, automaticallyassign related data addresses in said address space to a correspondingset of identifiable storage locations storing related data, therebyoptimizing accessibility of said related data via said data addressspace.
 31. The data storage system of claim 30, wherein said relateddata addresses comprise consecutive data addresses.
 32. The data storagesystem of claim 30, wherein said related data comprisestemporally-related data.
 33. The data storage system of claim 30,wherein said related data comprises data elements forming part of a samedata object.
 34. The data storage system of claim 30, wherein at leastsome of said related data is accessible post address space optimizationvia a single data read request.
 35. The data storage system of claim 27,wherein said optimization instructions comprise liberating instructionsthat, when implemented, automatically liberate a new contiguous block ofdata addresses in said address space by assigning new data addresses toone or more previously assigned contiguous data address blocksencompassed by said new contiguous block of data addresses.
 36. The datastorage system of claim 35, wherein said liberating instructions resultsin a writable data size increase available via a single write request.